Jul 24 22:15:02.498424 ip-10-0-143-53 systemd[1]: Starting Kubernetes Kubelet... Jul 24 22:15:02.888931 ip-10-0-143-53 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 22:15:02.888931 ip-10-0-143-53 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 22:15:02.888931 ip-10-0-143-53 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 22:15:02.888931 ip-10-0-143-53 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 22:15:02.888931 ip-10-0-143-53 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 22:15:02.889575 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.888931 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 22:15:02.894686 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894671 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:15:02.894686 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894686 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894690 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894693 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894697 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894700 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894703 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894705 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894708 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894711 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894713 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894716 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894719 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894723 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894726 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894734 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894737 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894740 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894743 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894745 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:15:02.894753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894748 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894751 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894754 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894757 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894760 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894763 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894766 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894769 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894771 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894774 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894776 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894779 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894782 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894784 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894787 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894790 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894792 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894795 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894797 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894800 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:15:02.895222 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894803 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894805 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894808 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894810 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894813 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894815 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894818 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894820 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894823 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894825 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894828 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894830 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894832 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894834 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894838 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894840 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894843 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894845 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894848 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894851 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:15:02.895697 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894853 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894856 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894858 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894861 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894864 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894866 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894869 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894871 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894874 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894876 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894878 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894882 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894886 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894888 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894892 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894895 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894898 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894901 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894906 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:15:02.896177 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894909 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894911 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894914 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894916 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894919 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894921 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894924 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894928 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894930 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894933 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894935 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894938 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894940 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894943 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894945 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894948 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894951 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894953 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894955 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.894958 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:15:02.896626 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895667 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895676 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895681 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895684 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895686 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895689 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895692 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895695 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895698 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895703 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895705 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895708 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895711 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895713 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895716 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895718 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895721 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895723 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895726 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:15:02.897093 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895730 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895733 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895736 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895739 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895741 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895744 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895746 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895749 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895752 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895754 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895757 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895759 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895762 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895764 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895767 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895769 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895771 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895774 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895776 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895779 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:15:02.897575 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895781 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895784 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895786 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895789 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895792 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895795 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895797 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895800 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895802 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895805 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895807 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895810 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895812 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895814 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895817 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895819 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895821 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895824 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895826 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:15:02.898053 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895828 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895832 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895835 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895840 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895843 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895846 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895849 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895852 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895855 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895857 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895859 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895862 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895865 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895867 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895870 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895873 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895876 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895878 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895881 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:15:02.898534 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895883 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895886 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895888 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895890 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895893 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895895 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895897 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895900 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895902 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895905 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895907 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895909 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895912 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895914 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895917 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895920 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895922 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895925 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895927 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895930 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:15:02.899014 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895935 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.895938 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896306 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896311 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896314 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896317 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896319 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896322 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896324 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896326 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896329 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896332 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896334 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896337 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896339 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896342 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896345 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896348 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896352 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:15:02.899509 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896358 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896361 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896363 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896366 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896369 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896371 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896374 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896376 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896379 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896381 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896384 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896386 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896388 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896392 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896395 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896398 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896401 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896404 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896406 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896409 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:15:02.900051 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896411 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896414 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896416 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896418 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896422 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896424 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896427 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896429 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896431 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896434 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896436 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896439 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896442 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896445 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896447 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896449 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896452 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896454 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896456 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:15:02.900594 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896459 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896462 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896464 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896468 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896471 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896473 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896477 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896481 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896483 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896486 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896488 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896491 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896493 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896496 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896498 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896501 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896503 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896506 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896509 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:15:02.901048 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896511 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896514 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896516 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896518 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896521 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896523 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896527 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896537 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896540 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896542 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896545 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896547 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896549 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896552 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896555 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896557 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896560 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896562 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896565 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896568 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:15:02.901514 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896571 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896574 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896576 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.896579 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896649 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896656 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896664 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896669 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896672 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896675 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896679 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896684 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896688 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896691 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896694 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896697 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896700 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896703 2554 flags.go:64] FLAG: --cgroup-root="" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896707 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896711 2554 flags.go:64] FLAG: --client-ca-file="" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896713 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896716 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896720 2554 flags.go:64] FLAG: --cluster-domain="" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896722 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 22:15:02.902015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896725 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896728 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896732 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896735 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896738 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896741 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896744 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896748 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896751 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896754 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896757 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896760 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896764 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896767 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896770 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896773 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896775 2554 flags.go:64] FLAG: --enable-server="true" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896778 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896782 2554 flags.go:64] FLAG: --event-burst="100" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896785 2554 flags.go:64] FLAG: --event-qps="50" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896788 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896791 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896794 2554 flags.go:64] FLAG: --eviction-hard="" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896797 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 22:15:02.902613 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896800 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896803 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896806 2554 flags.go:64] FLAG: --eviction-soft="" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896809 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896812 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896815 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896817 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896820 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896823 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896825 2554 flags.go:64] FLAG: --feature-gates="" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896829 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896832 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896835 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896838 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896841 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896845 2554 flags.go:64] FLAG: --help="false" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896848 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-143-53.ec2.internal" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896851 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896854 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896857 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896860 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896863 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896866 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 22:15:02.903194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896868 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896871 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896874 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896876 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896879 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896882 2554 flags.go:64] FLAG: --kube-reserved="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896885 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896888 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896891 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896894 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896896 2554 flags.go:64] FLAG: --lock-file="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896900 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896903 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896906 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896911 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896913 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896916 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896918 2554 flags.go:64] FLAG: --logging-format="text" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896921 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896924 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896927 2554 flags.go:64] FLAG: --manifest-url="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896929 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896933 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896936 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896941 2554 flags.go:64] FLAG: --max-pods="110" Jul 24 22:15:02.903783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896944 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896947 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896950 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896953 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896956 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896959 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896962 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896968 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896971 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896974 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896977 2554 flags.go:64] FLAG: --pod-cidr="" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896980 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896984 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896987 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896990 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.896997 2554 flags.go:64] FLAG: --port="10250" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897000 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897003 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0452ed727218dc457" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897008 2554 flags.go:64] FLAG: --qos-reserved="" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897011 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897014 2554 flags.go:64] FLAG: --register-node="true" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897016 2554 flags.go:64] FLAG: --register-with-taints="" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897020 2554 flags.go:64] FLAG: --registry-burst="10" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897022 2554 flags.go:64] FLAG: --registry-qps="5" Jul 24 22:15:02.904384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897025 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897028 2554 flags.go:64] FLAG: --reserved-memory="" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897031 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897034 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897037 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897040 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897042 2554 flags.go:64] FLAG: --runonce="false" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897045 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897049 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897052 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897055 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897058 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897060 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897063 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897066 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897069 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897072 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897074 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897077 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897080 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897083 2554 flags.go:64] FLAG: --system-cgroups="" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897085 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897090 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897093 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897097 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 22:15:02.904971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897100 2554 flags.go:64] FLAG: --tls-min-version="" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897104 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897107 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897110 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897113 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897115 2554 flags.go:64] FLAG: --v="2" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897119 2554 flags.go:64] FLAG: --version="false" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897123 2554 flags.go:64] FLAG: --vmodule="" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897126 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.897129 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897249 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897253 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897257 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897260 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897262 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897267 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897270 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897273 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897276 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897279 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897282 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897284 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:15:02.905612 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897287 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897290 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897293 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897296 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897298 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897301 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897304 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897306 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897309 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897312 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897315 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897318 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897323 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897326 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897329 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897332 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897335 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897337 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897340 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:15:02.906174 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897342 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897344 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897347 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897349 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897352 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897354 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897358 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897360 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897363 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897365 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897368 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897371 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897373 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897376 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897378 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897380 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897383 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897385 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897388 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:15:02.906635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897390 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897392 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897395 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897397 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897400 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897403 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897406 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897408 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897411 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897413 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897415 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897418 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897420 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897423 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897425 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897428 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897430 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897434 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897436 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897440 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:15:02.907114 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897442 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897445 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897449 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897452 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897455 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897458 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897461 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897465 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897468 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897471 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897473 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897476 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897478 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897481 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897483 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897486 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897488 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897490 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897493 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:15:02.907610 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897496 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897498 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897501 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897503 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897505 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897508 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897510 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897512 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897515 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.897517 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.898063 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.905074 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.905088 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.905747 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.905754 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 22:15:02.908061 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905813 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905819 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905822 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905825 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905828 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905831 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905833 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905836 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905839 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905842 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905844 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905847 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905849 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905852 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905854 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905857 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905860 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905863 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905865 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905868 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:15:02.908460 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905871 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905873 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905876 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905878 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905881 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905883 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905886 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905889 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905891 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905896 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905900 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905903 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905906 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905909 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905911 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905914 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905916 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905919 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905922 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905924 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:15:02.908981 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905927 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905929 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905932 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905934 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905937 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905939 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905942 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905944 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905947 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905950 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905953 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905955 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905958 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905960 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905963 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905965 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905967 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905970 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905972 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905975 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:15:02.909483 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905977 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905981 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905984 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905988 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905992 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905995 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.905998 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906000 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906003 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906005 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906008 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906010 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906013 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906015 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906018 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906020 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906023 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906025 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906027 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:15:02.909965 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906030 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906032 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906035 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906038 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906041 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906043 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906045 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906048 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906050 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906053 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906055 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906057 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906060 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906062 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906065 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906068 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906070 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906073 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906075 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:15:02.910441 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906078 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.906083 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906219 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906225 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906228 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906230 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906233 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906236 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906238 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906241 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906243 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906245 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906248 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906250 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906253 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 22:15:02.910903 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906255 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906258 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906261 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906263 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906266 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906269 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906272 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906275 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906278 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906281 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906284 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906286 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906288 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906291 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906293 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906295 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906298 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906300 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906304 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 22:15:02.911309 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906307 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906310 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906313 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906316 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906319 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906321 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906324 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906327 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906329 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906332 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906334 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906337 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906340 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906343 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906345 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906348 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906351 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906353 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906356 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906358 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 22:15:02.911753 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906360 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906363 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906365 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906367 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906370 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906372 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906375 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906377 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906379 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906382 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906385 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906387 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906390 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906392 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906395 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906397 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906399 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906402 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906404 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 22:15:02.912247 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906407 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906409 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906412 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906414 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906416 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906419 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906421 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906424 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906426 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906429 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906434 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906436 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906439 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906441 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906444 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906446 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906448 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906451 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906453 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906456 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 22:15:02.912713 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906459 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906461 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906464 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906466 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906469 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906471 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906473 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:02.906476 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.906481 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.906572 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.908499 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.909810 2554 server.go:1019] "Starting client certificate rotation" Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.909909 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 22:15:02.913212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.909943 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 22:15:02.913551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.912334 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 22:15:02.913870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.913856 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 22:15:02.926299 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.926276 2554 log.go:25] "Validated CRI v1 runtime API" Jul 24 22:15:02.928283 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.928265 2554 log.go:25] "Validated CRI v1 image API" Jul 24 22:15:02.929742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.929716 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 22:15:02.932021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.931996 2554 fs.go:135] Filesystem UUIDs: map[6a93fea2-134d-4022-8850-1599f2b67442:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f26718fc-bd16-4b22-9cbe-74ef449a5957:/dev/nvme0n1p4] Jul 24 22:15:02.932077 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.932022 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 22:15:02.938005 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.937888 2554 manager.go:216] Machine: {Timestamp:2026-07-24 22:15:02.936508356 +0000 UTC m=+0.323326396 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098605 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2294c03a93e7e16e1f7bba581fe69e SystemUUID:ec2294c0-3a93-e7e1-6e1f-7bba581fe69e BootID:1f765183-6b8d-4300-a0f8-56368b0d8374 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e0:57:ea:af:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e0:57:ea:af:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:34:bf:9b:6c:3f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 22:15:02.938005 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.937997 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 22:15:02.938174 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.938104 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 22:15:02.941017 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.940991 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 22:15:02.941190 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.941022 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-53.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 22:15:02.941876 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.941865 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 22:15:02.941940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.941881 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 22:15:02.941940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.941892 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 22:15:02.943506 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.943493 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 22:15:02.944757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.944745 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 22:15:02.948003 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.947990 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 24 22:15:02.948145 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.948134 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 22:15:02.948839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.948827 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 24 22:15:02.948891 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.948845 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 22:15:02.948891 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.948865 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 22:15:02.948891 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.948878 2554 kubelet.go:398] "Adding apiserver pod source" Jul 24 22:15:02.948891 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.948891 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 22:15:02.949289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.949275 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 22:15:02.949363 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.949299 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 22:15:02.951863 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.951845 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 22:15:02.952255 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.952239 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 22:15:02.953718 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.953702 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 22:15:02.953792 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.953740 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 22:15:02.954386 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954374 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 22:15:02.954449 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954393 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 22:15:02.954449 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954402 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 22:15:02.954449 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954410 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 22:15:02.954449 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954422 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 22:15:02.954449 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954431 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 22:15:02.954641 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954630 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 22:15:02.954689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954645 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 22:15:02.954689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954657 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 22:15:02.954689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954666 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 22:15:02.954689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954678 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 22:15:02.954813 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.954690 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 22:15:02.955501 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.955491 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 22:15:02.955549 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.955504 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 22:15:02.958487 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.958471 2554 server.go:1268] "Started kubelet" Jul 24 22:15:02.958599 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.958553 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 22:15:02.958683 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.958639 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 22:15:02.958735 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.958713 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 22:15:02.959251 ip-10-0-143-53 systemd[1]: Started Kubernetes Kubelet. Jul 24 22:15:02.960390 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.960369 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 22:15:02.961028 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.961014 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 24 22:15:02.965039 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.965014 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-53.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 22:15:02.965323 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.965301 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 22:15:02.965445 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.965426 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 22:15:02.966087 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.966067 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 22:15:02.967340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.967230 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 22:15:02.967340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.967248 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 22:15:02.967723 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.967317 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:02.967965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.967942 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 22:15:02.967965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.967958 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 22:15:02.968094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968077 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 22:15:02.968166 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968108 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 24 22:15:02.968166 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968114 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 24 22:15:02.968551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968521 2554 factory.go:55] Registering systemd factory Jul 24 22:15:02.968551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968550 2554 factory.go:223] Registration of the systemd container factory successfully Jul 24 22:15:02.968775 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968763 2554 factory.go:153] Registering CRI-O factory Jul 24 22:15:02.968838 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968776 2554 factory.go:223] Registration of the crio container factory successfully Jul 24 22:15:02.968838 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968823 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 22:15:02.968911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968843 2554 factory.go:103] Registering Raw factory Jul 24 22:15:02.968911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.968854 2554 manager.go:1196] Started watching for new ooms in manager Jul 24 22:15:02.969496 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.969485 2554 manager.go:349] Starting recovery of all containers Jul 24 22:15:02.969758 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.965736 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-53.ec2.internal.18c559902550328e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-53.ec2.internal,UID:ip-10-0-143-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-53.ec2.internal,},FirstTimestamp:2026-07-24 22:15:02.958445198 +0000 UTC m=+0.345263234,LastTimestamp:2026-07-24 22:15:02.958445198 +0000 UTC m=+0.345263234,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-53.ec2.internal,}" Jul 24 22:15:02.969983 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.969962 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 22:15:02.970077 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.970058 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-53.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 22:15:02.979947 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.979930 2554 manager.go:354] Recovery completed Jul 24 22:15:02.983845 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.983832 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:02.985553 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.985531 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:02.985627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.985568 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:02.985627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.985580 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:02.986163 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986135 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 22:15:02.986163 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986164 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 22:15:02.986269 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986179 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 24 22:15:02.986269 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986179 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:02.986929 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986914 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:02.987000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986942 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:02.987000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.986954 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:02.987383 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.987371 2554 policy_none.go:49] "None policy: Start" Jul 24 22:15:02.987419 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.987391 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 22:15:02.987419 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.987405 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 22:15:02.988786 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.988775 2554 policy_none.go:47] "Start" Jul 24 22:15:02.995011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:02.994485 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5ln8b" Jul 24 22:15:02.996732 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:02.996668 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-53.ec2.internal.18c5599026edd48f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-53.ec2.internal,UID:ip-10-0-143-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-53.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-53.ec2.internal,},FirstTimestamp:2026-07-24 22:15:02.985553039 +0000 UTC m=+0.372371076,LastTimestamp:2026-07-24 22:15:02.985553039 +0000 UTC m=+0.372371076,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-53.ec2.internal,}" Jul 24 22:15:03.004607 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.004546 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-53.ec2.internal.18c5599026ee28d9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-53.ec2.internal,UID:ip-10-0-143-53.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-53.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-53.ec2.internal,},FirstTimestamp:2026-07-24 22:15:02.985574617 +0000 UTC m=+0.372392655,LastTimestamp:2026-07-24 22:15:02.985574617 +0000 UTC m=+0.372392655,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-53.ec2.internal,}" Jul 24 22:15:03.005358 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.005340 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5ln8b" Jul 24 22:15:03.023687 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.023673 2554 manager.go:337] "Starting Device Plugin manager" Jul 24 22:15:03.023763 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.023699 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 22:15:03.023763 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.023708 2554 server.go:87] "Starting device plugin registration server" Jul 24 22:15:03.023936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.023924 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 22:15:03.023988 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.023937 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 22:15:03.024031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.024009 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 22:15:03.024087 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.024076 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 22:15:03.024133 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.024089 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 22:15:03.025649 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.025483 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 22:15:03.025744 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.025690 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.124397 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.124374 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:03.125926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.125261 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:03.125926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.125294 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:03.125926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.125313 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:03.125926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.125342 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.133826 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.133811 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.133898 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.133833 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-53.ec2.internal\": node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.150082 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.150039 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.168285 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.168261 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 22:15:03.169629 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.169612 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 22:15:03.169700 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.169646 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 22:15:03.169700 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.169665 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 22:15:03.169787 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.169699 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 22:15:03.171952 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.171933 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:03.250878 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.250845 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.270218 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.270191 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal"] Jul 24 22:15:03.270317 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.270255 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:03.271754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.271739 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:03.271836 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.271770 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:03.271836 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.271785 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:03.273233 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.273220 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:03.273364 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.273346 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.273448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.273377 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:03.275662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.275644 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:03.275804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.275677 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:03.275804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.275696 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:03.275804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.275646 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:03.275804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.275772 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:03.275804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.275784 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:03.277173 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.277144 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.277235 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.277183 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 22:15:03.277830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.277813 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientMemory" Jul 24 22:15:03.277918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.277843 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 22:15:03.277918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.277854 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeHasSufficientPID" Jul 24 22:15:03.293683 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.293659 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-53.ec2.internal\" not found" node="ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.297511 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.297495 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-53.ec2.internal\" not found" node="ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.351165 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.351114 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.370984 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.370961 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/25b6a6685b2afdc6382cf415895a2655-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal\" (UID: \"25b6a6685b2afdc6382cf415895a2655\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.371073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.370985 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25b6a6685b2afdc6382cf415895a2655-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal\" (UID: \"25b6a6685b2afdc6382cf415895a2655\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.371073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.371008 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5b643551357dc02beb09e72ad5e5fed-config\") pod \"kube-apiserver-proxy-ip-10-0-143-53.ec2.internal\" (UID: \"b5b643551357dc02beb09e72ad5e5fed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.451374 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.451320 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.471733 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.471709 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/25b6a6685b2afdc6382cf415895a2655-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal\" (UID: \"25b6a6685b2afdc6382cf415895a2655\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.471824 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.471739 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25b6a6685b2afdc6382cf415895a2655-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal\" (UID: \"25b6a6685b2afdc6382cf415895a2655\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.471824 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.471762 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5b643551357dc02beb09e72ad5e5fed-config\") pod \"kube-apiserver-proxy-ip-10-0-143-53.ec2.internal\" (UID: \"b5b643551357dc02beb09e72ad5e5fed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.471824 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.471815 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/25b6a6685b2afdc6382cf415895a2655-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal\" (UID: \"25b6a6685b2afdc6382cf415895a2655\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.471943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.471815 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/25b6a6685b2afdc6382cf415895a2655-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal\" (UID: \"25b6a6685b2afdc6382cf415895a2655\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.471943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.471847 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5b643551357dc02beb09e72ad5e5fed-config\") pod \"kube-apiserver-proxy-ip-10-0-143-53.ec2.internal\" (UID: \"b5b643551357dc02beb09e72ad5e5fed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.551906 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.551884 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.596437 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.596408 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.600372 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.600356 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.652284 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.652257 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.758465 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.758399 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-53.ec2.internal\" not found" Jul 24 22:15:03.828432 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.828407 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:03.842198 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.842179 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:03.868039 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.868017 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.881290 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.881269 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 22:15:03.882413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.882401 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" Jul 24 22:15:03.894457 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.894438 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 22:15:03.909711 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.909691 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 22:15:03.909842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.909823 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 22:15:03.909842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.909828 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 22:15:03.909932 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.909828 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 22:15:03.951100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.951080 2554 apiserver.go:52] "Watching apiserver" Jul 24 22:15:03.968127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.968104 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 22:15:03.973103 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.973083 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 22:15:03.974362 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.974331 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-95k6d","openshift-multus/network-metrics-daemon-kpxxn","openshift-network-diagnostics/network-check-target-bw5sl","openshift-ovn-kubernetes/ovnkube-node-rqpg4","kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw","openshift-cluster-node-tuning-operator/tuned-9c7k4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal","openshift-multus/multus-additional-cni-plugins-f4tnc","openshift-multus/multus-cxdjj","openshift-network-operator/iptables-alerter-fqgt6","kube-system/global-pull-secret-syncer-xvlrl","kube-system/konnectivity-agent-ld27j","openshift-dns/node-resolver-6848f"] Jul 24 22:15:03.976046 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.976028 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:03.977306 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.977280 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:03.977388 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.977366 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:03.978614 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.978599 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:03.978662 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.978645 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:03.978769 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.978752 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hgp47\"" Jul 24 22:15:03.979412 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.979396 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 22:15:03.979412 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.979404 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 22:15:03.979540 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.979433 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.980035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.980019 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:03.983844 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.983058 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:03.983844 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.983673 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 22:15:03.984046 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.984028 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 22:15:03.984277 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.984261 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 22:15:03.985191 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.984610 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:03.985191 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.984897 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 22:15:03.985361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.985194 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 22:15:03.985361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.985253 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5c2mb\"" Jul 24 22:15:03.985361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.985312 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 22:15:03.985361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.985328 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.986546 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.986523 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:03.988140 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.988122 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cxdjj" Jul 24 22:15:03.988913 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.988898 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 22:15:03.989001 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.988931 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z7f8v\"" Jul 24 22:15:03.989224 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.989211 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:15:03.989278 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.989227 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 22:15:03.989507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.989493 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.989620 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.989604 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:03.990110 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.990092 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6v9qb\"" Jul 24 22:15:03.990569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.990549 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 22:15:03.990666 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.990576 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 22:15:03.991248 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.991227 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-87bsr\"" Jul 24 22:15:03.991347 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.991255 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:03.991347 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:03.991315 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:03.991921 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.991904 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 22:15:03.992248 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.992212 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qfdvw\"" Jul 24 22:15:03.992718 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.992700 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.993056 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.993036 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:03.993507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.993484 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:15:03.994295 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994216 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 22:15:03.994295 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994264 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.994446 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994303 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.994446 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994320 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dvp2c\"" Jul 24 22:15:03.994446 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994400 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 22:15:03.994604 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994506 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 22:15:03.994682 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.994666 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:03.996984 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.996965 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 22:15:03.997067 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.996967 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2jwq2\"" Jul 24 22:15:03.997342 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.997326 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 22:15:03.998141 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.998121 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 22:15:03.999102 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.999082 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kgvmh\"" Jul 24 22:15:03.999194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:03.999129 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 22:15:04.008594 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.008510 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 22:10:03 +0000 UTC" deadline="2028-03-25 12:21:05.385958868 +0000 UTC" Jul 24 22:15:04.008594 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.008568 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14630h6m1.377395301s" Jul 24 22:15:04.029737 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.029715 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9h2r8" Jul 24 22:15:04.041115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.041098 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9h2r8" Jul 24 22:15:04.070209 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.070191 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 22:15:04.074930 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.074913 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpcmx\" (UniqueName: \"kubernetes.io/projected/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-kube-api-access-jpcmx\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.074988 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.074937 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-ovn\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.074988 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.074966 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-run\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075062 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.074987 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-var-lib-kubelet\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075062 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075004 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-registration-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.075062 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075027 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-system-cni-dir\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.075181 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075066 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-dbus\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.075181 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075106 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:04.075181 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:04.075181 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075176 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-tuned\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075192 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-system-cni-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075234 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-k8s-cni-cncf-io\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075278 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkcd4\" (UniqueName: \"kubernetes.io/projected/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-kube-api-access-hkcd4\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.075413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075316 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-serviceca\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.075413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075352 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-run-netns\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.075413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075377 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-conf-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075422 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075444 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-host\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075463 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-slash\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075479 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-hostroot\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075505 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-multus-certs\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075522 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n75lc\" (UniqueName: \"kubernetes.io/projected/a537a1d5-cc48-4106-85bb-5799059981b2-kube-api-access-n75lc\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075547 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-socket-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075567 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-var-lib-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075583 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-modprobe-d\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075599 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysconfig\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075615 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-kubernetes\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075630 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-host\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075662 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-socket-dir-parent\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075683 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d89mx\" (UniqueName: \"kubernetes.io/projected/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-kube-api-access-d89mx\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075698 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-etc-selinux\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075746 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvncf\" (UniqueName: \"kubernetes.io/projected/08821459-00c8-4cca-991b-e092689381e3-kube-api-access-dvncf\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075770 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvnlb\" (UniqueName: \"kubernetes.io/projected/3429398f-e14d-41cc-aa66-70b22fa7e963-kube-api-access-mvnlb\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075798 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075832 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-device-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075879 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075897 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ghkw\" (UniqueName: \"kubernetes.io/projected/a690e8b5-9861-49ee-965b-23443f6a7887-kube-api-access-6ghkw\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075931 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-run-ovn-kubernetes\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.075965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1f74a32-1b37-4ae4-aa33-aceb8751560c-tmp\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.076416 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.075969 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-etc-kubernetes\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.076416 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.076000 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-env-overrides\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.076674 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.076651 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovn-node-metrics-cert\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.076748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.076731 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-cnibin\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.076799 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.076767 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.076931 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.076887 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a690e8b5-9861-49ee-965b-23443f6a7887-host-slash\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.077025 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077005 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-systemd\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.077116 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077054 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysctl-d\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.077116 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077102 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-cni-bin\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.077280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077151 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-kubelet-config\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.077280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077198 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a537a1d5-cc48-4106-85bb-5799059981b2-hosts-file\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.077280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077246 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-systemd-units\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.077280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077276 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-cni-multus\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.077441 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077297 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0edcb77e-4272-4cc6-b46e-c90755fa671a-konnectivity-ca\") pod \"konnectivity-agent-ld27j\" (UID: \"0edcb77e-4272-4cc6-b46e-c90755fa671a\") " pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.077441 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077317 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-systemd\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.077904 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077868 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-cni-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.078318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.077910 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-os-release\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.078448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-netns\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.078448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078380 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-cni-bin\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.078448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078413 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.078448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078445 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-sys\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.078715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078526 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6nxn\" (UniqueName: \"kubernetes.io/projected/c1f74a32-1b37-4ae4-aa33-aceb8751560c-kube-api-access-h6nxn\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.078715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078561 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-sys-fs\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.078715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078597 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.078715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078665 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-kubelet\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.078913 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078716 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-etc-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.078913 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078764 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovnkube-script-lib\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.078913 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078813 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-lib-modules\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.078913 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078856 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-kubelet\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.078913 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078885 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537a1d5-cc48-4106-85bb-5799059981b2-tmp-dir\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078913 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-cni-netd\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.078953 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0edcb77e-4272-4cc6-b46e-c90755fa671a-agent-certs\") pod \"konnectivity-agent-ld27j\" (UID: \"0edcb77e-4272-4cc6-b46e-c90755fa671a\") " pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079001 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-os-release\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079034 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079071 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-log-socket\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079096 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdstk\" (UniqueName: \"kubernetes.io/projected/da976ab5-355f-45a4-ad40-3c763c57bcf8-kube-api-access-pdstk\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079132 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysctl-conf\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.079267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079162 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-cnibin\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.079662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079301 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-cni-binary-copy\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.079662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079384 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-daemon-config\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.079662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079416 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-node-log\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.079662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079442 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovnkube-config\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.079662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079475 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a690e8b5-9861-49ee-965b-23443f6a7887-iptables-alerter-script\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.079662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.079505 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.180490 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180452 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n75lc\" (UniqueName: \"kubernetes.io/projected/a537a1d5-cc48-4106-85bb-5799059981b2-kube-api-access-n75lc\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180497 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-socket-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180524 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-var-lib-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180552 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-modprobe-d\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180579 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysconfig\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180653 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-socket-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180656 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-var-lib-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180698 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-modprobe-d\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.180728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180713 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysconfig\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180739 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-kubernetes\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180776 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-host\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180792 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-socket-dir-parent\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180809 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d89mx\" (UniqueName: \"kubernetes.io/projected/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-kube-api-access-d89mx\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-etc-selinux\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180888 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-host\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180905 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-etc-selinux\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180904 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-kubernetes\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180935 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvncf\" (UniqueName: \"kubernetes.io/projected/08821459-00c8-4cca-991b-e092689381e3-kube-api-access-dvncf\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180970 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvnlb\" (UniqueName: \"kubernetes.io/projected/3429398f-e14d-41cc-aa66-70b22fa7e963-kube-api-access-mvnlb\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.180938 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-socket-dir-parent\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181000 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181031 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-device-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181039 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181063 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181083 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-device-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.181126 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181096 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ghkw\" (UniqueName: \"kubernetes.io/projected/a690e8b5-9861-49ee-965b-23443f6a7887-kube-api-access-6ghkw\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181122 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-run-ovn-kubernetes\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181147 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1f74a32-1b37-4ae4-aa33-aceb8751560c-tmp\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181196 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-run-ovn-kubernetes\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181195 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-etc-kubernetes\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181243 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-etc-kubernetes\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181246 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-env-overrides\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181263 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181277 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovn-node-metrics-cert\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181303 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-cnibin\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181331 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a690e8b5-9861-49ee-965b-23443f6a7887-host-slash\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181388 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-systemd\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181413 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysctl-d\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181398 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-cnibin\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-cni-bin\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181464 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-kubelet-config\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.181936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a537a1d5-cc48-4106-85bb-5799059981b2-hosts-file\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181489 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-systemd\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181513 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-systemd-units\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181535 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181569 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-cni-multus\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181578 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-cni-bin\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181587 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a537a1d5-cc48-4106-85bb-5799059981b2-hosts-file\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181595 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0edcb77e-4272-4cc6-b46e-c90755fa671a-konnectivity-ca\") pod \"konnectivity-agent-ld27j\" (UID: \"0edcb77e-4272-4cc6-b46e-c90755fa671a\") " pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181644 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-systemd-units\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181648 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-systemd\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181638 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-kubelet-config\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181641 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysctl-d\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181538 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a690e8b5-9861-49ee-965b-23443f6a7887-host-slash\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181712 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-systemd\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-env-overrides\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181751 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-cni-multus\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181779 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-cni-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181809 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-os-release\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.182841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181837 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-netns\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181862 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-cni-bin\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181886 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181909 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-sys\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181936 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-os-release\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181935 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6nxn\" (UniqueName: \"kubernetes.io/projected/c1f74a32-1b37-4ae4-aa33-aceb8751560c-kube-api-access-h6nxn\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181959 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-cni-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181980 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.181998 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-cni-bin\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182013 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-sys\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182002 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-sys-fs\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182046 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182047 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-netns\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182068 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182112 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-kubelet\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182112 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-sys-fs\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182140 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-etc-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182175 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0edcb77e-4272-4cc6-b46e-c90755fa671a-konnectivity-ca\") pod \"konnectivity-agent-ld27j\" (UID: \"0edcb77e-4272-4cc6-b46e-c90755fa671a\") " pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.183902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182183 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovnkube-script-lib\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182189 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-etc-openvswitch\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182234 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-lib-modules\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182259 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-kubelet\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182302 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-kubelet\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182383 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-lib-modules\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182391 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537a1d5-cc48-4106-85bb-5799059981b2-tmp-dir\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182389 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-var-lib-kubelet\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182434 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-cni-netd\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182462 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0edcb77e-4272-4cc6-b46e-c90755fa671a-agent-certs\") pod \"konnectivity-agent-ld27j\" (UID: \"0edcb77e-4272-4cc6-b46e-c90755fa671a\") " pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182494 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-os-release\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182502 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-cni-netd\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182532 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182559 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-log-socket\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182579 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-os-release\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182689 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a537a1d5-cc48-4106-85bb-5799059981b2-tmp-dir\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182583 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdstk\" (UniqueName: \"kubernetes.io/projected/da976ab5-355f-45a4-ad40-3c763c57bcf8-kube-api-access-pdstk\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.184933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182798 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovnkube-script-lib\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182855 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysctl-conf\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182962 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-sysctl-conf\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.182989 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-cnibin\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183011 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-log-socket\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-cni-binary-copy\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183044 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-daemon-config\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183063 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-cnibin\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183069 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-node-log\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183099 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovnkube-config\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183121 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a690e8b5-9861-49ee-965b-23443f6a7887-iptables-alerter-script\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183145 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183168 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpcmx\" (UniqueName: \"kubernetes.io/projected/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-kube-api-access-jpcmx\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183192 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-ovn\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183215 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-run\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183270 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-var-lib-kubelet\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183267 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.186115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183297 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-registration-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183327 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-system-cni-dir\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183349 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-node-log\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183354 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-dbus\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183399 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183430 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183456 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-tuned\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-system-cni-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183507 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-k8s-cni-cncf-io\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkcd4\" (UniqueName: \"kubernetes.io/projected/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-kube-api-access-hkcd4\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183566 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-serviceca\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-run-netns\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183621 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-conf-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183647 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183670 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-host\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183716 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-slash\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183745 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-hostroot\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.187079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-multus-certs\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183848 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a690e8b5-9861-49ee-965b-23443f6a7887-iptables-alerter-script\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183880 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-multus-certs\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.183949 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/08821459-00c8-4cca-991b-e092689381e3-cni-binary-copy\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.183978 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.184069 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:15:04.684046703 +0000 UTC m=+2.070864753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184245 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-system-cni-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184304 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-host-run-k8s-cni-cncf-io\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184342 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-run-ovn\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184395 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-run\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184425 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovnkube-config\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184451 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c1f74a32-1b37-4ae4-aa33-aceb8751560c-var-lib-kubelet\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184514 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184538 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-dbus\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.184545 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184571 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/08821459-00c8-4cca-991b-e092689381e3-system-cni-dir\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184592 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-slash\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.188035 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.184615 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:04.684596957 +0000 UTC m=+2.071414994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184553 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-host\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184666 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da976ab5-355f-45a4-ad40-3c763c57bcf8-host-run-netns\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184715 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-conf-dir\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184761 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-registration-dir\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.184786 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-hostroot\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.185036 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-serviceca\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.185210 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c1f74a32-1b37-4ae4-aa33-aceb8751560c-tmp\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.185355 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-cni-binary-copy\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.185444 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da976ab5-355f-45a4-ad40-3c763c57bcf8-ovn-node-metrics-cert\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.185556 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0edcb77e-4272-4cc6-b46e-c90755fa671a-agent-certs\") pod \"konnectivity-agent-ld27j\" (UID: \"0edcb77e-4272-4cc6-b46e-c90755fa671a\") " pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.187596 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c1f74a32-1b37-4ae4-aa33-aceb8751560c-etc-tuned\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.188625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.187929 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-multus-daemon-config\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.193873 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.193851 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:04.193873 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.193875 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:04.194033 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.193885 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:04.194033 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.193935 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:04.693921668 +0000 UTC m=+2.080739693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:04.195723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.195696 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvnlb\" (UniqueName: \"kubernetes.io/projected/3429398f-e14d-41cc-aa66-70b22fa7e963-kube-api-access-mvnlb\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:04.199660 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.199632 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvncf\" (UniqueName: \"kubernetes.io/projected/08821459-00c8-4cca-991b-e092689381e3-kube-api-access-dvncf\") pod \"multus-additional-cni-plugins-f4tnc\" (UID: \"08821459-00c8-4cca-991b-e092689381e3\") " pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.208548 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.208525 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d89mx\" (UniqueName: \"kubernetes.io/projected/65765a0f-dd2d-4bca-a6cf-7bd77ead7425-kube-api-access-d89mx\") pod \"multus-cxdjj\" (UID: \"65765a0f-dd2d-4bca-a6cf-7bd77ead7425\") " pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.209145 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.209126 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkcd4\" (UniqueName: \"kubernetes.io/projected/bb5e7144-b3a8-4cb4-b306-7d1e2d730586-kube-api-access-hkcd4\") pod \"aws-ebs-csi-driver-node-v5snw\" (UID: \"bb5e7144-b3a8-4cb4-b306-7d1e2d730586\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.209837 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.209817 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6nxn\" (UniqueName: \"kubernetes.io/projected/c1f74a32-1b37-4ae4-aa33-aceb8751560c-kube-api-access-h6nxn\") pod \"tuned-9c7k4\" (UID: \"c1f74a32-1b37-4ae4-aa33-aceb8751560c\") " pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.209917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.209823 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpcmx\" (UniqueName: \"kubernetes.io/projected/8eb6f6a5-436c-4a1b-9333-ca67a6e50c53-kube-api-access-jpcmx\") pod \"node-ca-95k6d\" (UID: \"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53\") " pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.209917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.209876 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdstk\" (UniqueName: \"kubernetes.io/projected/da976ab5-355f-45a4-ad40-3c763c57bcf8-kube-api-access-pdstk\") pod \"ovnkube-node-rqpg4\" (UID: \"da976ab5-355f-45a4-ad40-3c763c57bcf8\") " pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.209917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.209878 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ghkw\" (UniqueName: \"kubernetes.io/projected/a690e8b5-9861-49ee-965b-23443f6a7887-kube-api-access-6ghkw\") pod \"iptables-alerter-fqgt6\" (UID: \"a690e8b5-9861-49ee-965b-23443f6a7887\") " pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.213685 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.213664 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n75lc\" (UniqueName: \"kubernetes.io/projected/a537a1d5-cc48-4106-85bb-5799059981b2-kube-api-access-n75lc\") pod \"node-resolver-6848f\" (UID: \"a537a1d5-cc48-4106-85bb-5799059981b2\") " pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.239034 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.239000 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5b643551357dc02beb09e72ad5e5fed.slice/crio-732a9883f1faaf6490fbf74b8675363e3315c436cd7ef7db465193169ad91c53 WatchSource:0}: Error finding container 732a9883f1faaf6490fbf74b8675363e3315c436cd7ef7db465193169ad91c53: Status 404 returned error can't find the container with id 732a9883f1faaf6490fbf74b8675363e3315c436cd7ef7db465193169ad91c53 Jul 24 22:15:04.239581 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.239562 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b6a6685b2afdc6382cf415895a2655.slice/crio-4ad24ca2ab6b77f6113022d7f31e708641bb80f9ae1bb68550401d910322e2d4 WatchSource:0}: Error finding container 4ad24ca2ab6b77f6113022d7f31e708641bb80f9ae1bb68550401d910322e2d4: Status 404 returned error can't find the container with id 4ad24ca2ab6b77f6113022d7f31e708641bb80f9ae1bb68550401d910322e2d4 Jul 24 22:15:04.245939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.245922 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 22:15:04.297950 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.297892 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-95k6d" Jul 24 22:15:04.306353 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.306324 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb6f6a5_436c_4a1b_9333_ca67a6e50c53.slice/crio-076127090e25f1c246b22ea5c66dc51b1176fc77315f808d65308d6ce9b41c4b WatchSource:0}: Error finding container 076127090e25f1c246b22ea5c66dc51b1176fc77315f808d65308d6ce9b41c4b: Status 404 returned error can't find the container with id 076127090e25f1c246b22ea5c66dc51b1176fc77315f808d65308d6ce9b41c4b Jul 24 22:15:04.312860 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.312841 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:04.318588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.318569 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" Jul 24 22:15:04.320611 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.320463 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda976ab5_355f_45a4_ad40_3c763c57bcf8.slice/crio-6fcefda98456dbcfb1ff6aa125277255712d80476a599cfd0cfcaf0a1f504107 WatchSource:0}: Error finding container 6fcefda98456dbcfb1ff6aa125277255712d80476a599cfd0cfcaf0a1f504107: Status 404 returned error can't find the container with id 6fcefda98456dbcfb1ff6aa125277255712d80476a599cfd0cfcaf0a1f504107 Jul 24 22:15:04.326350 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.326329 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5e7144_b3a8_4cb4_b306_7d1e2d730586.slice/crio-118f31e3c5cd7483b817ea179111005537274b0f869f8418149c762c81e3c149 WatchSource:0}: Error finding container 118f31e3c5cd7483b817ea179111005537274b0f869f8418149c762c81e3c149: Status 404 returned error can't find the container with id 118f31e3c5cd7483b817ea179111005537274b0f869f8418149c762c81e3c149 Jul 24 22:15:04.338867 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.338842 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" Jul 24 22:15:04.345607 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.345585 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f74a32_1b37_4ae4_aa33_aceb8751560c.slice/crio-dd133b912c07c3d2f3a2a0e3c32c00f7f4ff8febe4ccc9b86185b5302fbf72c6 WatchSource:0}: Error finding container dd133b912c07c3d2f3a2a0e3c32c00f7f4ff8febe4ccc9b86185b5302fbf72c6: Status 404 returned error can't find the container with id dd133b912c07c3d2f3a2a0e3c32c00f7f4ff8febe4ccc9b86185b5302fbf72c6 Jul 24 22:15:04.352868 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.352849 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" Jul 24 22:15:04.360894 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.360876 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08821459_00c8_4cca_991b_e092689381e3.slice/crio-baa15204e4334285a720d098275c0274826f1842a046d1f134c59f3c05f1f169 WatchSource:0}: Error finding container baa15204e4334285a720d098275c0274826f1842a046d1f134c59f3c05f1f169: Status 404 returned error can't find the container with id baa15204e4334285a720d098275c0274826f1842a046d1f134c59f3c05f1f169 Jul 24 22:15:04.368840 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.368817 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cxdjj" Jul 24 22:15:04.376159 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.376140 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65765a0f_dd2d_4bca_a6cf_7bd77ead7425.slice/crio-62c27979d91041523c5745a5b4145d00535f53a596abacf5fa0d7bf0cb540662 WatchSource:0}: Error finding container 62c27979d91041523c5745a5b4145d00535f53a596abacf5fa0d7bf0cb540662: Status 404 returned error can't find the container with id 62c27979d91041523c5745a5b4145d00535f53a596abacf5fa0d7bf0cb540662 Jul 24 22:15:04.383361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.383344 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fqgt6" Jul 24 22:15:04.389542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.389523 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:04.390811 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.390790 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda690e8b5_9861_49ee_965b_23443f6a7887.slice/crio-9e9f7d4ad70a7c833d46317587e4c861202b7bf7503eef87fc58a41d713220ff WatchSource:0}: Error finding container 9e9f7d4ad70a7c833d46317587e4c861202b7bf7503eef87fc58a41d713220ff: Status 404 returned error can't find the container with id 9e9f7d4ad70a7c833d46317587e4c861202b7bf7503eef87fc58a41d713220ff Jul 24 22:15:04.407640 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.407623 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6848f" Jul 24 22:15:04.415760 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:04.415740 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda537a1d5_cc48_4106_85bb_5799059981b2.slice/crio-4481d30b4215999d2c970619bb77e49982ce0483d5d7ef81c06d10ac55ddfb2f WatchSource:0}: Error finding container 4481d30b4215999d2c970619bb77e49982ce0483d5d7ef81c06d10ac55ddfb2f: Status 404 returned error can't find the container with id 4481d30b4215999d2c970619bb77e49982ce0483d5d7ef81c06d10ac55ddfb2f Jul 24 22:15:04.417286 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.417266 2554 kuberuntime_manager.go:1459] "Unhandled Error" err="container dns-node-resolver start failed in pod node-resolver-6848f_openshift-dns(a537a1d5-cc48-4106-85bb-5799059981b2): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 24 22:15:04.417364 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.417300 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-dns/node-resolver-6848f" podUID="a537a1d5-cc48-4106-85bb-5799059981b2" Jul 24 22:15:04.526419 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.526390 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:04.687450 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.687358 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:04.687450 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.687412 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:04.687656 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.687571 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:04.687656 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.687635 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:05.687614588 +0000 UTC m=+3.074432628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:04.687798 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.687703 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:04.687798 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.687738 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:15:05.687726267 +0000 UTC m=+3.074544311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:04.789685 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:04.788617 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:04.789685 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.788810 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:04.789685 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.788833 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:04.789685 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.788846 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:04.789685 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:04.788910 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:05.788888252 +0000 UTC m=+3.175706284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:05.043384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.043261 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:10:04 +0000 UTC" deadline="2028-03-21 13:27:51.407256414 +0000 UTC" Jul 24 22:15:05.043384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.043325 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14535h12m46.363936754s" Jul 24 22:15:05.173583 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.173551 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:05.173838 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.173687 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:05.185739 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.185686 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" event={"ID":"c1f74a32-1b37-4ae4-aa33-aceb8751560c","Type":"ContainerStarted","Data":"dd133b912c07c3d2f3a2a0e3c32c00f7f4ff8febe4ccc9b86185b5302fbf72c6"} Jul 24 22:15:05.187428 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.187398 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" event={"ID":"25b6a6685b2afdc6382cf415895a2655","Type":"ContainerStarted","Data":"4ad24ca2ab6b77f6113022d7f31e708641bb80f9ae1bb68550401d910322e2d4"} Jul 24 22:15:05.190656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.190629 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" event={"ID":"b5b643551357dc02beb09e72ad5e5fed","Type":"ContainerStarted","Data":"732a9883f1faaf6490fbf74b8675363e3315c436cd7ef7db465193169ad91c53"} Jul 24 22:15:05.203155 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.203126 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ld27j" event={"ID":"0edcb77e-4272-4cc6-b46e-c90755fa671a","Type":"ContainerStarted","Data":"d09f0389cd8a5472b9281e95bcf119cd2e3c71e6706e25c1ad785716b9cecb68"} Jul 24 22:15:05.219022 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.218987 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cxdjj" event={"ID":"65765a0f-dd2d-4bca-a6cf-7bd77ead7425","Type":"ContainerStarted","Data":"62c27979d91041523c5745a5b4145d00535f53a596abacf5fa0d7bf0cb540662"} Jul 24 22:15:05.222954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.222923 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerStarted","Data":"baa15204e4334285a720d098275c0274826f1842a046d1f134c59f3c05f1f169"} Jul 24 22:15:05.226458 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.226432 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" event={"ID":"bb5e7144-b3a8-4cb4-b306-7d1e2d730586","Type":"ContainerStarted","Data":"118f31e3c5cd7483b817ea179111005537274b0f869f8418149c762c81e3c149"} Jul 24 22:15:05.229354 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.229330 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"6fcefda98456dbcfb1ff6aa125277255712d80476a599cfd0cfcaf0a1f504107"} Jul 24 22:15:05.233056 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.233030 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-95k6d" event={"ID":"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53","Type":"ContainerStarted","Data":"076127090e25f1c246b22ea5c66dc51b1176fc77315f808d65308d6ce9b41c4b"} Jul 24 22:15:05.235458 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.235432 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6848f" event={"ID":"a537a1d5-cc48-4106-85bb-5799059981b2","Type":"ContainerStarted","Data":"4481d30b4215999d2c970619bb77e49982ce0483d5d7ef81c06d10ac55ddfb2f"} Jul 24 22:15:05.240739 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.237193 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"dns-node-resolver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:233b9dfa42a292bdae9563a0cfb304f006bf4cf1f75229e9b3147c1019c74859\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-dns/node-resolver-6848f" podUID="a537a1d5-cc48-4106-85bb-5799059981b2" Jul 24 22:15:05.242027 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.242001 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fqgt6" event={"ID":"a690e8b5-9861-49ee-965b-23443f6a7887","Type":"ContainerStarted","Data":"9e9f7d4ad70a7c833d46317587e4c861202b7bf7503eef87fc58a41d713220ff"} Jul 24 22:15:05.269327 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.269302 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:05.696987 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.696867 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:05.696987 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.696960 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:05.697218 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.697111 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:05.697218 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.697173 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:07.697153829 +0000 UTC m=+5.083971858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:05.697669 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.697647 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:05.697788 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.697713 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:15:07.697697769 +0000 UTC m=+5.084515794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:05.797348 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:05.797312 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:05.797519 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.797466 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:05.797519 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.797489 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:05.797519 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.797502 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:05.797671 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:05.797559 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:07.7975395 +0000 UTC m=+5.184357544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:06.043953 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:06.043838 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 22:10:04 +0000 UTC" deadline="2028-01-19 09:20:12.357081457 +0000 UTC" Jul 24 22:15:06.043953 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:06.043903 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13043h5m6.313182672s" Jul 24 22:15:06.170655 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:06.170621 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:06.170820 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:06.170757 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:06.171168 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:06.171147 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:06.171388 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:06.171364 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:06.220933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:06.220904 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:06.724810 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:06.724781 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 22:15:07.173047 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:07.172969 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:07.173554 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.173098 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:07.712612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:07.712576 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:07.712770 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:07.712638 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:07.712770 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.712707 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:07.712841 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.712780 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:07.712841 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.712793 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:15:11.712770346 +0000 UTC m=+9.099588376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:07.712841 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.712837 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:11.71282058 +0000 UTC m=+9.099638621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:07.813580 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:07.813535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:07.813763 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.813713 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:07.813763 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.813735 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:07.813763 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.813750 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:07.813943 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:07.813812 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:11.813792421 +0000 UTC m=+9.200610472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:08.170316 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:08.170213 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:08.170490 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:08.170368 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:08.170763 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:08.170744 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:08.170864 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:08.170826 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:09.170288 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:09.170257 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:09.170764 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:09.170401 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:10.170319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:10.170278 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:10.170798 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:10.170426 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:10.170798 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:10.170499 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:10.170798 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:10.170628 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:11.170728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:11.170173 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:11.170728 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.170305 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:11.752619 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:11.752334 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:11.752619 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:11.752400 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:11.752619 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.752522 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:11.752619 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.752546 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:11.752619 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.752600 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:15:19.752578509 +0000 UTC m=+17.139396550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:11.752619 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.752624 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:19.752612956 +0000 UTC m=+17.139430985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:11.853877 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:11.853785 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:11.854095 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.853998 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:11.854095 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.854028 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:11.854095 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.854043 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:11.854095 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:11.854113 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:19.854093941 +0000 UTC m=+17.240911982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:12.170585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:12.170502 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:12.170743 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:12.170681 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:12.171155 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:12.171110 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:12.171383 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:12.171210 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:13.179534 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:13.179502 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:13.180259 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:13.180207 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:14.171015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:14.170411 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:14.171015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:14.170458 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:14.171015 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:14.170578 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:14.171015 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:14.170694 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:15.170331 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:15.170300 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:15.170832 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:15.170413 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:16.170098 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:16.170057 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:16.170288 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:16.170055 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:16.170288 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:16.170216 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:16.170386 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:16.170301 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:17.170823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:17.170738 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:17.171295 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:17.170877 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:18.170085 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:18.170046 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:18.170283 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:18.170193 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:18.170283 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:18.170236 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:18.170368 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:18.170320 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:19.170301 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:19.170268 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:19.170758 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.170391 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:19.817319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:19.817283 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:19.817516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:19.817329 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:19.817516 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.817456 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:19.817516 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.817458 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:19.817672 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.817523 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:35.817504148 +0000 UTC m=+33.204322187 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:19.817672 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.817541 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:15:35.817532082 +0000 UTC m=+33.204350124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:19.918597 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:19.918560 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:19.918787 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.918724 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:19.918787 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.918745 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:19.918787 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.918754 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:19.918907 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:19.918802 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:35.918790556 +0000 UTC m=+33.305608580 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:20.170093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:20.170000 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:20.170093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:20.170028 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:20.170379 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:20.170134 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:20.170746 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:20.170386 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:21.171370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:21.170877 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:21.171370 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:21.171002 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:22.169959 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:22.169927 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:22.170172 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:22.169939 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:22.170172 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:22.170069 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:22.170172 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:22.170113 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:23.171468 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:23.171432 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:23.171864 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:23.171548 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:24.170355 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:24.170314 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:24.170550 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:24.170318 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:24.170550 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:24.170441 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:24.170550 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:24.170532 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:25.170639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.170558 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:25.170989 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:25.170690 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:25.290914 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.290879 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" event={"ID":"c1f74a32-1b37-4ae4-aa33-aceb8751560c","Type":"ContainerStarted","Data":"cbb55e5975556e26a553a7dfb8a9d5635f0a48f7317201cd6424423b1aa05da3"} Jul 24 22:15:25.292303 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.292281 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" event={"ID":"b5b643551357dc02beb09e72ad5e5fed","Type":"ContainerStarted","Data":"6ff0da5eee534a946a849d374cf16300ceb0fcb1836cb4ada3e3662838a86aab"} Jul 24 22:15:25.293641 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.293617 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cxdjj" event={"ID":"65765a0f-dd2d-4bca-a6cf-7bd77ead7425","Type":"ContainerStarted","Data":"d87a9581c8c254c4934e83ffe462abda06a3158ed55855ebf3dc09995081195d"} Jul 24 22:15:25.323187 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.323113 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9c7k4" podStartSLOduration=0.65401653 podStartE2EDuration="21.323098764s" totalImagesPullingTime="20.669082234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:25.322502553 +0000 UTC m=+22.709320599" watchObservedRunningTime="2026-07-24 22:15:25.323098764 +0000 UTC m=+22.709916809" Jul 24 22:15:25.341840 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.341800 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-53.ec2.internal" podStartSLOduration=21.341787229 podStartE2EDuration="21.341787229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:25.341515944 +0000 UTC m=+22.728333989" watchObservedRunningTime="2026-07-24 22:15:25.341787229 +0000 UTC m=+22.728605275" Jul 24 22:15:25.364511 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:25.364472 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cxdjj" podStartSLOduration=0.710388954 podStartE2EDuration="21.364458577s" totalImagesPullingTime="20.654069623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:25.363928018 +0000 UTC m=+22.750746066" watchObservedRunningTime="2026-07-24 22:15:25.364458577 +0000 UTC m=+22.751276622" Jul 24 22:15:26.170106 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.169906 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:26.170342 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.169986 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:26.170342 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:26.170251 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:26.170454 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:26.170387 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:26.300103 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.300060 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"cc09918c0dff3f04f7f775ca909d6b33785c57385607843ff604c0cc8e3476f4"} Jul 24 22:15:26.300757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.300139 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"3c5881275539a04913c6e8a534d8e6e1a39db688a4a3737c6a5a6eb9921c3257"} Jul 24 22:15:26.300757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.300193 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"ca30683f9b768dffe45c287f0b497d57ed4f7f016c22636c2c680e04c88bec27"} Jul 24 22:15:26.300757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.300223 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"6b339d17adb2751b214b0e66bc330e62c6fd3b2dea016632dab248c4fe66fffc"} Jul 24 22:15:26.300757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.300260 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"f6e748b6f8aceae6a1faed15bb66c8517f3fb8217b044b802107bed378590465"} Jul 24 22:15:26.300757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.300288 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"7c12ecd61996945609d954a949b06230469b4ecc5130d2b6e19706b519f5e36a"} Jul 24 22:15:26.301773 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.301750 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-95k6d" event={"ID":"8eb6f6a5-436c-4a1b-9333-ca67a6e50c53","Type":"ContainerStarted","Data":"8811e7250eba018f81a14a88b700b37c6af84b5deeeb7717d12d2e613ace7e19"} Jul 24 22:15:26.303169 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.303094 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6848f" event={"ID":"a537a1d5-cc48-4106-85bb-5799059981b2","Type":"ContainerStarted","Data":"3cd69cd02cc7432af79f1b56f95ccbec930feaef16578d54911a6e44797a5ca3"} Jul 24 22:15:26.304499 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.304478 2554 generic.go:356] "Generic (PLEG): container finished" podID="25b6a6685b2afdc6382cf415895a2655" containerID="447285e8d47410a6299dea1563e99d7916cd53bf0d8e47f089c12b99b5d55304" exitCode=0 Jul 24 22:15:26.304592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.304539 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" event={"ID":"25b6a6685b2afdc6382cf415895a2655","Type":"ContainerDied","Data":"447285e8d47410a6299dea1563e99d7916cd53bf0d8e47f089c12b99b5d55304"} Jul 24 22:15:26.305981 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.305961 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ld27j" event={"ID":"0edcb77e-4272-4cc6-b46e-c90755fa671a","Type":"ContainerStarted","Data":"5a79af38b8e407fd7598aa5320fa2577709c3191d905906fa5640fcf35cdfcc8"} Jul 24 22:15:26.307459 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.307439 2554 generic.go:356] "Generic (PLEG): container finished" podID="08821459-00c8-4cca-991b-e092689381e3" containerID="311335c67d8974190643a58965e710864488bcc75922ea50b5780ba917c15522" exitCode=0 Jul 24 22:15:26.307543 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.307500 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerDied","Data":"311335c67d8974190643a58965e710864488bcc75922ea50b5780ba917c15522"} Jul 24 22:15:26.308873 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.308854 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" event={"ID":"bb5e7144-b3a8-4cb4-b306-7d1e2d730586","Type":"ContainerStarted","Data":"86e15cadb85faf75a9d984015642c81d8e1fa3417567c7250211171410002454"} Jul 24 22:15:26.317657 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.317620 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-95k6d" podStartSLOduration=1.611063937 podStartE2EDuration="22.31760998s" totalImagesPullingTime="20.706546043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:26.317416008 +0000 UTC m=+23.704234053" watchObservedRunningTime="2026-07-24 22:15:26.31760998 +0000 UTC m=+23.704428026" Jul 24 22:15:26.328543 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.328524 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:26.328939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.328926 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:26.334659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.334628 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6848f" podStartSLOduration=22.334618235 podStartE2EDuration="22.334618235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-24 22:15:26.334296589 +0000 UTC m=+23.721114634" watchObservedRunningTime="2026-07-24 22:15:26.334618235 +0000 UTC m=+23.721436280" Jul 24 22:15:26.378722 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.378678 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ld27j" podStartSLOduration=6.216089478 podStartE2EDuration="22.378659372s" totalImagesPullingTime="16.162569894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:26.378318219 +0000 UTC m=+23.765136267" watchObservedRunningTime="2026-07-24 22:15:26.378659372 +0000 UTC m=+23.765477418" Jul 24 22:15:26.999484 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:26.999328 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 22:15:27.038787 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.038626 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T22:15:26.999476658Z","UUID":"abd768bf-2b7f-40cb-b354-c87447491098","Handler":null,"Name":"","Endpoint":""} Jul 24 22:15:27.040338 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.040316 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 22:15:27.040461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.040361 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 22:15:27.170425 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.170356 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:27.170620 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:27.170484 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:27.313138 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.313107 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" event={"ID":"bb5e7144-b3a8-4cb4-b306-7d1e2d730586","Type":"ContainerStarted","Data":"041233dc1f780dc3036336fce80c5385d22399aa73d4ba0596884138e1f22efe"} Jul 24 22:15:27.314740 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.314715 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fqgt6" event={"ID":"a690e8b5-9861-49ee-965b-23443f6a7887","Type":"ContainerStarted","Data":"3a02aa621f9f96c572f2a475d7d007700003c96e76dcdb17eb5afbb3455012e0"} Jul 24 22:15:27.316784 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.316763 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" event={"ID":"25b6a6685b2afdc6382cf415895a2655","Type":"ContainerStarted","Data":"6242014487edd0133b6771fd54fbc42d8935c6c84d43f923e9920e27235f8449"} Jul 24 22:15:27.317193 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.317175 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:27.317623 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.317606 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ld27j" Jul 24 22:15:27.329349 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.329308 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fqgt6" podStartSLOduration=2.728226281 podStartE2EDuration="23.329295945s" totalImagesPullingTime="20.601069664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:27.328924134 +0000 UTC m=+24.715742181" watchObservedRunningTime="2026-07-24 22:15:27.329295945 +0000 UTC m=+24.716113990" Jul 24 22:15:27.357939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:27.357901 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-53.ec2.internal" podStartSLOduration=23.357888873 podStartE2EDuration="23.357888873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:27.357760768 +0000 UTC m=+24.744578814" watchObservedRunningTime="2026-07-24 22:15:27.357888873 +0000 UTC m=+24.744706918" Jul 24 22:15:28.170269 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:28.170235 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:28.170514 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:28.170273 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:28.170514 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:28.170359 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:28.170832 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:28.170806 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:28.322419 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:28.322387 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" event={"ID":"bb5e7144-b3a8-4cb4-b306-7d1e2d730586","Type":"ContainerStarted","Data":"86fe4f0a694dced9f3e105f1d0ab06f485ca6bfc53662cee6ebefc892d31ed53"} Jul 24 22:15:29.170907 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:29.170873 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:29.171079 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:29.171008 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:29.332713 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:29.332674 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"f313dff1c2561d590d0035b1a18015e15ee8cb943857102cdedd10f16cec8ebc"} Jul 24 22:15:29.349890 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:29.349848 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v5snw" podStartSLOduration=2.324406137 podStartE2EDuration="25.349834368s" totalImagesPullingTime="23.025428231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:29.349766443 +0000 UTC m=+26.736584503" watchObservedRunningTime="2026-07-24 22:15:29.349834368 +0000 UTC m=+26.736652414" Jul 24 22:15:30.170596 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:30.170568 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:30.170804 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:30.170685 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:30.170804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:30.170730 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:30.170917 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:30.170843 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:31.170790 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.170525 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:31.171269 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:31.170860 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:31.338484 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.338450 2554 generic.go:356] "Generic (PLEG): container finished" podID="08821459-00c8-4cca-991b-e092689381e3" containerID="e0f81e8db71befc5811f516a5a4ff9aaf5216b002e471d6d9aaf8f2332b2819b" exitCode=0 Jul 24 22:15:31.338649 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.338531 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerDied","Data":"e0f81e8db71befc5811f516a5a4ff9aaf5216b002e471d6d9aaf8f2332b2819b"} Jul 24 22:15:31.342112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.342091 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" event={"ID":"da976ab5-355f-45a4-ad40-3c763c57bcf8","Type":"ContainerStarted","Data":"aee000ba8f24510882a24bcc112e547cbe60d02bca761e4dc4ff3dbfd7712668"} Jul 24 22:15:31.342376 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.342355 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:31.342482 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.342428 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:31.357887 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.357860 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:31.358072 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.358055 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:31.392432 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.392389 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" podStartSLOduration=6.303648161 podStartE2EDuration="27.392376301s" totalImagesPullingTime="21.08872814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:31.390559358 +0000 UTC m=+28.777377404" watchObservedRunningTime="2026-07-24 22:15:31.392376301 +0000 UTC m=+28.779194390" Jul 24 22:15:31.647236 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:31.647214 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:15:32.170103 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.170082 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:32.170292 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.170110 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:32.170292 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:32.170196 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:32.170375 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:32.170309 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:32.552349 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.552138 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bw5sl"] Jul 24 22:15:32.552850 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.552437 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:32.552850 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:32.552541 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:32.555427 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.555406 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kpxxn"] Jul 24 22:15:32.555535 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.555494 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:32.555535 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.555501 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xvlrl"] Jul 24 22:15:32.555620 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:32.555596 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:32.555655 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:32.555638 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:32.555718 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:32.555701 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:33.347473 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:33.347441 2554 generic.go:356] "Generic (PLEG): container finished" podID="08821459-00c8-4cca-991b-e092689381e3" containerID="f7007411f07e480ca4764a14e829b214a89fd78cffcc4db3573acd345681f0a5" exitCode=0 Jul 24 22:15:33.347654 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:33.347516 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerDied","Data":"f7007411f07e480ca4764a14e829b214a89fd78cffcc4db3573acd345681f0a5"} Jul 24 22:15:34.170132 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:34.170108 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:34.170132 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:34.170119 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:34.170132 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:34.170131 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:34.170603 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:34.170248 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:34.170603 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:34.170358 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:34.170603 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:34.170424 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:35.357107 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:35.357076 2554 generic.go:356] "Generic (PLEG): container finished" podID="08821459-00c8-4cca-991b-e092689381e3" containerID="fa60f9582750480c9807e32dc5e97a5a19ee9b5f7b00893f83c5f326f1426f0c" exitCode=0 Jul 24 22:15:35.357436 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:35.357132 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerDied","Data":"fa60f9582750480c9807e32dc5e97a5a19ee9b5f7b00893f83c5f326f1426f0c"} Jul 24 22:15:35.837357 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:35.837322 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:35.837551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:35.837368 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:35.837551 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.837477 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:35.837551 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.837477 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:35.837551 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.837542 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs podName:3429398f-e14d-41cc-aa66-70b22fa7e963 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:07.837522965 +0000 UTC m=+65.224341008 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs") pod "network-metrics-daemon-kpxxn" (UID: "3429398f-e14d-41cc-aa66-70b22fa7e963") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 22:15:35.837771 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.837560 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret podName:259c7f52-5c99-47cf-bd40-9aea3ea0fc6d nodeName:}" failed. No retries permitted until 2026-07-24 22:16:07.83755033 +0000 UTC m=+65.224368355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret") pod "global-pull-secret-syncer-xvlrl" (UID: "259c7f52-5c99-47cf-bd40-9aea3ea0fc6d") : object "kube-system"/"original-pull-secret" not registered Jul 24 22:15:35.938349 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:35.938316 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:35.938488 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.938467 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 22:15:35.938488 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.938485 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 22:15:35.938571 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.938494 2554 projected.go:196] Error preparing data for projected volume kube-api-access-m7gl8 for pod openshift-network-diagnostics/network-check-target-bw5sl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:35.938571 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:35.938541 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8 podName:d9f0a9b5-1255-470c-94a6-8fc07f32e6a0 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:07.938527732 +0000 UTC m=+65.325345755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m7gl8" (UniqueName: "kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8") pod "network-check-target-bw5sl" (UID: "d9f0a9b5-1255-470c-94a6-8fc07f32e6a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 22:15:36.170557 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:36.170527 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:36.170557 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:36.170549 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:36.170770 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:36.170541 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:36.170770 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:36.170657 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xvlrl" podUID="259c7f52-5c99-47cf-bd40-9aea3ea0fc6d" Jul 24 22:15:36.170770 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:36.170763 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kpxxn" podUID="3429398f-e14d-41cc-aa66-70b22fa7e963" Jul 24 22:15:36.170931 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:36.170822 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bw5sl" podUID="d9f0a9b5-1255-470c-94a6-8fc07f32e6a0" Jul 24 22:15:37.898529 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.898502 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-53.ec2.internal" event="NodeReady" Jul 24 22:15:37.898963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.898611 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 22:15:37.950937 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.950908 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7ln87"] Jul 24 22:15:37.954107 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.954085 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:37.954300 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.954278 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jr455"] Jul 24 22:15:37.956038 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.956013 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:37.958610 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.958587 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 22:15:37.958705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.958621 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wnkgk\"" Jul 24 22:15:37.958705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.958593 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 22:15:37.960185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.960171 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 22:15:37.961648 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.961618 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 22:15:37.961939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.961912 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 22:15:37.962074 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.962018 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ck9v8\"" Jul 24 22:15:37.965542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.965522 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7ln87"] Jul 24 22:15:37.967060 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:37.967040 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jr455"] Jul 24 22:15:38.056775 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.056695 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-tmp-dir\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.056775 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.056742 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wnh8\" (UniqueName: \"kubernetes.io/projected/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-kube-api-access-4wnh8\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.056958 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.056809 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1604448-3b43-4289-a7da-abfe3cb9b4c1-cert\") pod \"ingress-canary-jr455\" (UID: \"b1604448-3b43-4289-a7da-abfe3cb9b4c1\") " pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.056958 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.056841 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmdph\" (UniqueName: \"kubernetes.io/projected/b1604448-3b43-4289-a7da-abfe3cb9b4c1-kube-api-access-xmdph\") pod \"ingress-canary-jr455\" (UID: \"b1604448-3b43-4289-a7da-abfe3cb9b4c1\") " pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.056958 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.056867 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-config-volume\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.056958 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.056911 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.157551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.157524 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.157689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.157586 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-tmp-dir\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.157689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.157611 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wnh8\" (UniqueName: \"kubernetes.io/projected/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-kube-api-access-4wnh8\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.157689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.157641 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1604448-3b43-4289-a7da-abfe3cb9b4c1-cert\") pod \"ingress-canary-jr455\" (UID: \"b1604448-3b43-4289-a7da-abfe3cb9b4c1\") " pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.157892 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:38.157690 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:15:38.157892 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:38.157764 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls podName:3bd4a67d-4043-43f8-bdfd-cdb12975a5e8 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:38.657744994 +0000 UTC m=+36.044563030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls") pod "dns-default-7ln87" (UID: "3bd4a67d-4043-43f8-bdfd-cdb12975a5e8") : secret "dns-default-metrics-tls" not found Jul 24 22:15:38.157892 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.157684 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmdph\" (UniqueName: \"kubernetes.io/projected/b1604448-3b43-4289-a7da-abfe3cb9b4c1-kube-api-access-xmdph\") pod \"ingress-canary-jr455\" (UID: \"b1604448-3b43-4289-a7da-abfe3cb9b4c1\") " pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.157892 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.157874 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-config-volume\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.158094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.158071 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-tmp-dir\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.158469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.158446 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-config-volume\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.162856 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.162832 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1604448-3b43-4289-a7da-abfe3cb9b4c1-cert\") pod \"ingress-canary-jr455\" (UID: \"b1604448-3b43-4289-a7da-abfe3cb9b4c1\") " pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.169962 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.169935 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmdph\" (UniqueName: \"kubernetes.io/projected/b1604448-3b43-4289-a7da-abfe3cb9b4c1-kube-api-access-xmdph\") pod \"ingress-canary-jr455\" (UID: \"b1604448-3b43-4289-a7da-abfe3cb9b4c1\") " pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.170141 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.170126 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:15:38.170485 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.170466 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:15:38.170756 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.170737 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:15:38.174824 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.174799 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wnh8\" (UniqueName: \"kubernetes.io/projected/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-kube-api-access-4wnh8\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.175314 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.175295 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rtdx2\"" Jul 24 22:15:38.175408 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.175344 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 22:15:38.175408 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.175348 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 22:15:38.175614 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.175591 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-767d7\"" Jul 24 22:15:38.175716 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.175614 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 22:15:38.175716 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.175655 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 22:15:38.272068 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.272041 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jr455" Jul 24 22:15:38.662904 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:38.662866 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:38.663072 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:38.663052 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:15:38.663131 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:38.663120 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls podName:3bd4a67d-4043-43f8-bdfd-cdb12975a5e8 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:39.663105953 +0000 UTC m=+37.049923981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls") pod "dns-default-7ln87" (UID: "3bd4a67d-4043-43f8-bdfd-cdb12975a5e8") : secret "dns-default-metrics-tls" not found Jul 24 22:15:39.465616 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:39.465423 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jr455"] Jul 24 22:15:39.467927 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:39.467883 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1604448_3b43_4289_a7da_abfe3cb9b4c1.slice/crio-00413231316d520657439bd7784c30bf4dcb593252993fa939bfdb4fdf898340 WatchSource:0}: Error finding container 00413231316d520657439bd7784c30bf4dcb593252993fa939bfdb4fdf898340: Status 404 returned error can't find the container with id 00413231316d520657439bd7784c30bf4dcb593252993fa939bfdb4fdf898340 Jul 24 22:15:39.670883 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:39.670851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:39.671032 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:39.671002 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:15:39.671080 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:39.671070 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls podName:3bd4a67d-4043-43f8-bdfd-cdb12975a5e8 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:41.671054084 +0000 UTC m=+39.057872107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls") pod "dns-default-7ln87" (UID: "3bd4a67d-4043-43f8-bdfd-cdb12975a5e8") : secret "dns-default-metrics-tls" not found Jul 24 22:15:40.369658 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:40.369613 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jr455" event={"ID":"b1604448-3b43-4289-a7da-abfe3cb9b4c1","Type":"ContainerStarted","Data":"00413231316d520657439bd7784c30bf4dcb593252993fa939bfdb4fdf898340"} Jul 24 22:15:40.372561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:40.372533 2554 generic.go:356] "Generic (PLEG): container finished" podID="08821459-00c8-4cca-991b-e092689381e3" containerID="19ffc9f6156d933c8392d15e34f5fbe0126051f3f1e9f5608bdfe9480dce2bac" exitCode=0 Jul 24 22:15:40.372691 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:40.372576 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerDied","Data":"19ffc9f6156d933c8392d15e34f5fbe0126051f3f1e9f5608bdfe9480dce2bac"} Jul 24 22:15:41.378849 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:41.378769 2554 generic.go:356] "Generic (PLEG): container finished" podID="08821459-00c8-4cca-991b-e092689381e3" containerID="b9bdd549e134d4d7e30044d14b5aede1802a9065bd8145b50017e7278ca95dd6" exitCode=0 Jul 24 22:15:41.379257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:41.378853 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerDied","Data":"b9bdd549e134d4d7e30044d14b5aede1802a9065bd8145b50017e7278ca95dd6"} Jul 24 22:15:41.691310 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:41.691072 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:41.691447 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:41.691226 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:15:41.691505 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:41.691486 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls podName:3bd4a67d-4043-43f8-bdfd-cdb12975a5e8 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:45.691463384 +0000 UTC m=+43.078281429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls") pod "dns-default-7ln87" (UID: "3bd4a67d-4043-43f8-bdfd-cdb12975a5e8") : secret "dns-default-metrics-tls" not found Jul 24 22:15:42.383882 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:42.383798 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" event={"ID":"08821459-00c8-4cca-991b-e092689381e3","Type":"ContainerStarted","Data":"30410d0a2222a0fdb4979c71ae3317cc61428aeeb7d29acd9017d140b5a1b496"} Jul 24 22:15:42.385251 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:42.385228 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jr455" event={"ID":"b1604448-3b43-4289-a7da-abfe3cb9b4c1","Type":"ContainerStarted","Data":"4063b567a9d1e6cc4c11115630befcf6a48f4644ab6f414c38441c9b23608630"} Jul 24 22:15:42.415603 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:42.415562 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f4tnc" podStartSLOduration=11.785734183 podStartE2EDuration="38.415548669s" totalImagesPullingTime="26.629814486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:42.415379826 +0000 UTC m=+39.802197873" watchObservedRunningTime="2026-07-24 22:15:42.415548669 +0000 UTC m=+39.802366715" Jul 24 22:15:42.435945 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:42.435895 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jr455" podStartSLOduration=2.9775861040000002 podStartE2EDuration="5.435884413s" totalImagesPullingTime="2.458298309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:42.435588762 +0000 UTC m=+39.822406809" watchObservedRunningTime="2026-07-24 22:15:42.435884413 +0000 UTC m=+39.822702459" Jul 24 22:15:45.570674 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.570631 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mm46z"] Jul 24 22:15:45.605169 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.605123 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-858d756bd9-xqm8h"] Jul 24 22:15:45.605304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.605247 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.610209 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.610182 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 22:15:45.610209 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.610201 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 22:15:45.610392 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.610222 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 24 22:15:45.610392 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.610232 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 24 22:15:45.610392 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.610202 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hgv22\"" Jul 24 22:15:45.619265 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.619249 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mm46z"] Jul 24 22:15:45.619345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.619274 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-858d756bd9-xqm8h"] Jul 24 22:15:45.619345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.619333 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.622103 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.622087 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 24 22:15:45.623635 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623464 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 24 22:15:45.623635 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623509 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 24 22:15:45.623635 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623528 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 24 22:15:45.623635 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623552 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tkms8\"" Jul 24 22:15:45.623635 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623565 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 22:15:45.623911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623528 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 24 22:15:45.623911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.623509 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 22:15:45.663861 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.663832 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4"] Jul 24 22:15:45.687224 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.687206 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4"] Jul 24 22:15:45.687291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.687278 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.694868 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.694841 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 22:15:45.694868 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.694859 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 24 22:15:45.694978 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.694845 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 22:15:45.694978 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.694901 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 24 22:15:45.695096 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.695083 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xxpnc\"" Jul 24 22:15:45.718308 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718287 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36fd0bd6-9a6b-4676-844c-0802bce022a5-tmp\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.718397 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718322 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/36fd0bd6-9a6b-4676-844c-0802bce022a5-snapshots\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.718397 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718348 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36fd0bd6-9a6b-4676-844c-0802bce022a5-service-ca-bundle\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.718474 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718410 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:45.718474 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718462 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvtr6\" (UniqueName: \"kubernetes.io/projected/36fd0bd6-9a6b-4676-844c-0802bce022a5-kube-api-access-bvtr6\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.718564 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718488 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36fd0bd6-9a6b-4676-844c-0802bce022a5-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.718564 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.718505 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36fd0bd6-9a6b-4676-844c-0802bce022a5-serving-cert\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.718632 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.718581 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:15:45.718632 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.718627 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls podName:3bd4a67d-4043-43f8-bdfd-cdb12975a5e8 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:53.718613327 +0000 UTC m=+51.105431353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls") pod "dns-default-7ln87" (UID: "3bd4a67d-4043-43f8-bdfd-cdb12975a5e8") : secret "dns-default-metrics-tls" not found Jul 24 22:15:45.818855 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.818830 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36fd0bd6-9a6b-4676-844c-0802bce022a5-tmp\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.818998 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.818858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/36fd0bd6-9a6b-4676-844c-0802bce022a5-snapshots\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.818998 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.818878 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36fd0bd6-9a6b-4676-844c-0802bce022a5-service-ca-bundle\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.818998 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.818897 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.818998 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.818915 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ff0d475-2d6b-4854-8748-93f562053c6f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.819193 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819090 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.819193 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819130 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvtr6\" (UniqueName: \"kubernetes.io/projected/36fd0bd6-9a6b-4676-844c-0802bce022a5-kube-api-access-bvtr6\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.819289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819195 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-default-certificate\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.819289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819218 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.819289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819245 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-stats-auth\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.819289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819251 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36fd0bd6-9a6b-4676-844c-0802bce022a5-tmp\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.819289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819276 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq57f\" (UniqueName: \"kubernetes.io/projected/2ff0d475-2d6b-4854-8748-93f562053c6f-kube-api-access-pq57f\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.819494 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819322 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36fd0bd6-9a6b-4676-844c-0802bce022a5-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.819494 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819353 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36fd0bd6-9a6b-4676-844c-0802bce022a5-serving-cert\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.819494 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819384 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4lgq\" (UniqueName: \"kubernetes.io/projected/43c537e5-5787-4685-8933-340514153061-kube-api-access-x4lgq\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.819622 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819491 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36fd0bd6-9a6b-4676-844c-0802bce022a5-service-ca-bundle\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.819622 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819573 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/36fd0bd6-9a6b-4676-844c-0802bce022a5-snapshots\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.820011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.819986 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36fd0bd6-9a6b-4676-844c-0802bce022a5-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.823217 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.823171 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36fd0bd6-9a6b-4676-844c-0802bce022a5-serving-cert\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.829801 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.829779 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvtr6\" (UniqueName: \"kubernetes.io/projected/36fd0bd6-9a6b-4676-844c-0802bce022a5-kube-api-access-bvtr6\") pod \"insights-operator-5885fdb759-mm46z\" (UID: \"36fd0bd6-9a6b-4676-844c-0802bce022a5\") " pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.914159 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.914135 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-mm46z" Jul 24 22:15:45.920367 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920345 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-default-certificate\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.920435 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920376 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.920435 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920393 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-stats-auth\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.920435 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920420 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq57f\" (UniqueName: \"kubernetes.io/projected/2ff0d475-2d6b-4854-8748-93f562053c6f-kube-api-access-pq57f\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.920588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920452 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4lgq\" (UniqueName: \"kubernetes.io/projected/43c537e5-5787-4685-8933-340514153061-kube-api-access-x4lgq\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.920588 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.920484 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:15:45.920588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920490 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.920588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920515 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ff0d475-2d6b-4854-8748-93f562053c6f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.920588 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.920529 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:46.420515709 +0000 UTC m=+43.807333733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : secret "router-metrics-certs-default" not found Jul 24 22:15:45.920837 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.920629 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:46.420611758 +0000 UTC m=+43.807429804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : configmap references non-existent config key: service-ca.crt Jul 24 22:15:45.920837 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.920697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.920837 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.920803 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:45.920967 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:45.920857 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls podName:2ff0d475-2d6b-4854-8748-93f562053c6f nodeName:}" failed. No retries permitted until 2026-07-24 22:15:46.420843006 +0000 UTC m=+43.807661046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-b5ck4" (UID: "2ff0d475-2d6b-4854-8748-93f562053c6f") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:45.921127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.921110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ff0d475-2d6b-4854-8748-93f562053c6f-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.922576 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.922557 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-default-certificate\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.922772 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.922754 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-stats-auth\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:45.930863 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.930838 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq57f\" (UniqueName: \"kubernetes.io/projected/2ff0d475-2d6b-4854-8748-93f562053c6f-kube-api-access-pq57f\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:45.930929 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:45.930867 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4lgq\" (UniqueName: \"kubernetes.io/projected/43c537e5-5787-4685-8933-340514153061-kube-api-access-x4lgq\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:46.079968 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:46.079944 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-mm46z"] Jul 24 22:15:46.081402 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:46.081378 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fd0bd6_9a6b_4676_844c_0802bce022a5.slice/crio-e4977987df256c0e0c8ae8f00c16f710472fd0e637c22fff00a3c26d4a04ab4a WatchSource:0}: Error finding container e4977987df256c0e0c8ae8f00c16f710472fd0e637c22fff00a3c26d4a04ab4a: Status 404 returned error can't find the container with id e4977987df256c0e0c8ae8f00c16f710472fd0e637c22fff00a3c26d4a04ab4a Jul 24 22:15:46.395718 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:46.395684 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mm46z" event={"ID":"36fd0bd6-9a6b-4676-844c-0802bce022a5","Type":"ContainerStarted","Data":"e4977987df256c0e0c8ae8f00c16f710472fd0e637c22fff00a3c26d4a04ab4a"} Jul 24 22:15:46.424224 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:46.424198 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:46.424350 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:46.424234 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:46.424350 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:46.424291 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:46.424432 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:46.424355 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:46.424432 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:46.424421 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls podName:2ff0d475-2d6b-4854-8748-93f562053c6f nodeName:}" failed. No retries permitted until 2026-07-24 22:15:47.424398003 +0000 UTC m=+44.811216040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-b5ck4" (UID: "2ff0d475-2d6b-4854-8748-93f562053c6f") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:46.424432 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:46.424422 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:15:46.424538 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:46.424444 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:47.424433044 +0000 UTC m=+44.811251069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : configmap references non-existent config key: service-ca.crt Jul 24 22:15:46.424538 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:46.424494 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:47.424476684 +0000 UTC m=+44.811294737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : secret "router-metrics-certs-default" not found Jul 24 22:15:47.431680 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:47.431641 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:47.431710 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:47.431777 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:47.431796 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:47.431831 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:47.431872 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls podName:2ff0d475-2d6b-4854-8748-93f562053c6f nodeName:}" failed. No retries permitted until 2026-07-24 22:15:49.431851698 +0000 UTC m=+46.818669741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-b5ck4" (UID: "2ff0d475-2d6b-4854-8748-93f562053c6f") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:47.431887 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:49.431880344 +0000 UTC m=+46.818698368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : secret "router-metrics-certs-default" not found Jul 24 22:15:47.432042 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:47.431913 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:49.431895661 +0000 UTC m=+46.818713710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : configmap references non-existent config key: service-ca.crt Jul 24 22:15:48.402507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:48.402477 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mm46z" event={"ID":"36fd0bd6-9a6b-4676-844c-0802bce022a5","Type":"ContainerStarted","Data":"5417ef9add8a445e5039f8c0f52cc5fd771f2c2093b4d983b0aa9c27f3d27f86"} Jul 24 22:15:48.425730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:48.425682 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-mm46z" podStartSLOduration=1.265605968 podStartE2EDuration="3.425667374s" totalImagesPullingTime="2.160061406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:48.424352542 +0000 UTC m=+45.811170599" watchObservedRunningTime="2026-07-24 22:15:48.425667374 +0000 UTC m=+45.812485449" Jul 24 22:15:49.055744 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.055712 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wl6px"] Jul 24 22:15:49.086866 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.086837 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wl6px"] Jul 24 22:15:49.087016 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.086933 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.122117 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.122093 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 22:15:49.122342 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.122329 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5nftg\"" Jul 24 22:15:49.122544 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.122525 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 22:15:49.143421 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.143392 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.143534 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.143454 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/90cc2e83-c8f6-4f31-8bdc-389922945c35-crio-socket\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.143534 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.143486 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/90cc2e83-c8f6-4f31-8bdc-389922945c35-data-volume\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.143645 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.143534 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp67c\" (UniqueName: \"kubernetes.io/projected/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-api-access-tp67c\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.143645 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.143635 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244013 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.243986 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.244022 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/90cc2e83-c8f6-4f31-8bdc-389922945c35-crio-socket\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.244041 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/90cc2e83-c8f6-4f31-8bdc-389922945c35-data-volume\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.244067 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp67c\" (UniqueName: \"kubernetes.io/projected/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-api-access-tp67c\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244127 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.244122 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:15:49.244366 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.244210 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:49.744188482 +0000 UTC m=+47.131006508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : configmap "kube-rbac-proxy" not found Jul 24 22:15:49.244366 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.244280 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/90cc2e83-c8f6-4f31-8bdc-389922945c35-crio-socket\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244366 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.244301 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.244516 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.244424 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:15:49.244516 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.244475 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:49.744460476 +0000 UTC m=+47.131278517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : secret "insights-runtime-extractor-tls" not found Jul 24 22:15:49.244516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.244476 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/90cc2e83-c8f6-4f31-8bdc-389922945c35-data-volume\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.257357 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.257333 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp67c\" (UniqueName: \"kubernetes.io/projected/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-api-access-tp67c\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.445936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.445892 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:49.446129 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.445998 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:49.446129 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.446112 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:49.446287 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.446175 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:53.446111128 +0000 UTC m=+50.832929169 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : configmap references non-existent config key: service-ca.crt Jul 24 22:15:49.446287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.446263 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:49.446404 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.446310 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls podName:2ff0d475-2d6b-4854-8748-93f562053c6f nodeName:}" failed. No retries permitted until 2026-07-24 22:15:53.446298137 +0000 UTC m=+50.833116161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-b5ck4" (UID: "2ff0d475-2d6b-4854-8748-93f562053c6f") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:49.446404 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.446358 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:15:49.446404 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.446399 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:53.4463862 +0000 UTC m=+50.833204243 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : secret "router-metrics-certs-default" not found Jul 24 22:15:49.749689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.749605 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.749689 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.749663 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:49.749860 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.749750 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:15:49.749860 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.749752 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:15:49.749860 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.749818 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:50.749803483 +0000 UTC m=+48.136621507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : secret "insights-runtime-extractor-tls" not found Jul 24 22:15:49.749860 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:49.749831 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:50.749824857 +0000 UTC m=+48.136642880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : configmap "kube-rbac-proxy" not found Jul 24 22:15:49.886115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.886088 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5"] Jul 24 22:15:49.902618 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.902596 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5"] Jul 24 22:15:49.902758 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.902697 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" Jul 24 22:15:49.910284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.907407 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 22:15:49.910284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.908169 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 22:15:49.910284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.908536 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9m844\"" Jul 24 22:15:49.951287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:49.951262 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trpcb\" (UniqueName: \"kubernetes.io/projected/f4af5df1-f958-47f0-9304-3cb39f18cf42-kube-api-access-trpcb\") pod \"migrator-6865c9967-tzzf5\" (UID: \"f4af5df1-f958-47f0-9304-3cb39f18cf42\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" Jul 24 22:15:50.051794 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.051734 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trpcb\" (UniqueName: \"kubernetes.io/projected/f4af5df1-f958-47f0-9304-3cb39f18cf42-kube-api-access-trpcb\") pod \"migrator-6865c9967-tzzf5\" (UID: \"f4af5df1-f958-47f0-9304-3cb39f18cf42\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" Jul 24 22:15:50.061734 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.061712 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trpcb\" (UniqueName: \"kubernetes.io/projected/f4af5df1-f958-47f0-9304-3cb39f18cf42-kube-api-access-trpcb\") pod \"migrator-6865c9967-tzzf5\" (UID: \"f4af5df1-f958-47f0-9304-3cb39f18cf42\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" Jul 24 22:15:50.215738 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.215713 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" Jul 24 22:15:50.393372 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.393349 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5"] Jul 24 22:15:50.394761 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:15:50.394731 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4af5df1_f958_47f0_9304_3cb39f18cf42.slice/crio-5bea596e84962c83616f4249d89cad820332e7e5868832a5b545371d3495af10 WatchSource:0}: Error finding container 5bea596e84962c83616f4249d89cad820332e7e5868832a5b545371d3495af10: Status 404 returned error can't find the container with id 5bea596e84962c83616f4249d89cad820332e7e5868832a5b545371d3495af10 Jul 24 22:15:50.407626 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.407597 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" event={"ID":"f4af5df1-f958-47f0-9304-3cb39f18cf42","Type":"ContainerStarted","Data":"5bea596e84962c83616f4249d89cad820332e7e5868832a5b545371d3495af10"} Jul 24 22:15:50.758501 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.758477 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:50.758621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.758523 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:50.758621 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:50.758600 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:15:50.758703 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:50.758629 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:15:50.758703 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:50.758652 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:52.758637788 +0000 UTC m=+50.145455812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : configmap "kube-rbac-proxy" not found Jul 24 22:15:50.758703 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:50.758684 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:52.758668897 +0000 UTC m=+50.145486926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : secret "insights-runtime-extractor-tls" not found Jul 24 22:15:50.851823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:50.851802 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6848f_a537a1d5-cc48-4106-85bb-5799059981b2/dns-node-resolver/0.log" Jul 24 22:15:51.850747 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:51.850718 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-95k6d_8eb6f6a5-436c-4a1b-9333-ca67a6e50c53/node-ca/0.log" Jul 24 22:15:52.414327 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:52.414297 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" event={"ID":"f4af5df1-f958-47f0-9304-3cb39f18cf42","Type":"ContainerStarted","Data":"6631e6f69b2073d441661982273f4941fa48c73edcc7a8dd6f0509a3932e38f7"} Jul 24 22:15:52.414527 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:52.414357 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" event={"ID":"f4af5df1-f958-47f0-9304-3cb39f18cf42","Type":"ContainerStarted","Data":"98f8842429ea1e66c907c4ea4a70e963e728f0ecaa300d40c4b21ca06a11edb8"} Jul 24 22:15:52.432677 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:52.432633 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tzzf5" podStartSLOduration=1.87422526 podStartE2EDuration="3.432617953s" totalImagesPullingTime="1.558392693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:15:52.432351301 +0000 UTC m=+49.819169348" watchObservedRunningTime="2026-07-24 22:15:52.432617953 +0000 UTC m=+49.819435999" Jul 24 22:15:52.773388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:52.773325 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:52.773507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:52.773436 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:52.773507 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:52.773442 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:15:52.773507 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:52.773500 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:56.773483651 +0000 UTC m=+54.160301679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : configmap "kube-rbac-proxy" not found Jul 24 22:15:52.773665 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:52.773547 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:15:52.773665 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:52.773598 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:15:56.773585826 +0000 UTC m=+54.160403855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : secret "insights-runtime-extractor-tls" not found Jul 24 22:15:52.856427 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:52.856403 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr455_b1604448-3b43-4289-a7da-abfe3cb9b4c1/serve-healthcheck-canary/0.log" Jul 24 22:15:53.478166 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:53.478123 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:53.478327 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:53.478211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:15:53.478327 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:53.478242 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:15:53.478327 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.478282 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:01.478266244 +0000 UTC m=+58.865084271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : configmap references non-existent config key: service-ca.crt Jul 24 22:15:53.478457 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.478349 2554 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 24 22:15:53.478457 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.478353 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:53.478457 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.478398 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:01.478382416 +0000 UTC m=+58.865200455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : secret "router-metrics-certs-default" not found Jul 24 22:15:53.478457 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.478416 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls podName:2ff0d475-2d6b-4854-8748-93f562053c6f nodeName:}" failed. No retries permitted until 2026-07-24 22:16:01.478406447 +0000 UTC m=+58.865224473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-b5ck4" (UID: "2ff0d475-2d6b-4854-8748-93f562053c6f") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:15:53.780471 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:53.780397 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:15:53.780607 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.780533 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 22:15:53.780607 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:53.780583 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls podName:3bd4a67d-4043-43f8-bdfd-cdb12975a5e8 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:09.780569679 +0000 UTC m=+67.167387706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls") pod "dns-default-7ln87" (UID: "3bd4a67d-4043-43f8-bdfd-cdb12975a5e8") : secret "dns-default-metrics-tls" not found Jul 24 22:15:56.803207 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:56.803167 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:56.803677 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:15:56.803244 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:15:56.803677 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:56.803330 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:15:56.803677 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:56.803395 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:04.803377856 +0000 UTC m=+62.190195880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : secret "insights-runtime-extractor-tls" not found Jul 24 22:15:56.803677 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:56.803330 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 22:15:56.803677 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:15:56.803452 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:04.803439851 +0000 UTC m=+62.190257875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : configmap "kube-rbac-proxy" not found Jul 24 22:16:01.541248 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:01.541212 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:01.541792 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:01.541287 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:16:01.541792 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:01.541378 2554 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 24 22:16:01.541792 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:01.541425 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle podName:43c537e5-5787-4685-8933-340514153061 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:17.541397089 +0000 UTC m=+74.928215164 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle") pod "router-default-858d756bd9-xqm8h" (UID: "43c537e5-5787-4685-8933-340514153061") : configmap references non-existent config key: service-ca.crt Jul 24 22:16:01.541792 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:01.541453 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls podName:2ff0d475-2d6b-4854-8748-93f562053c6f nodeName:}" failed. No retries permitted until 2026-07-24 22:16:17.541443323 +0000 UTC m=+74.928261353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-b5ck4" (UID: "2ff0d475-2d6b-4854-8748-93f562053c6f") : secret "cluster-monitoring-operator-tls" not found Jul 24 22:16:01.541792 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:01.541517 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:01.543996 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:01.543974 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/43c537e5-5787-4685-8933-340514153061-metrics-certs\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:03.367726 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:03.367698 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rqpg4" Jul 24 22:16:04.867592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:04.867554 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:16:04.868049 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:04.867612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:16:04.868049 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:04.867715 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 22:16:04.868049 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:04.867791 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls podName:90cc2e83-c8f6-4f31-8bdc-389922945c35 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:20.867768643 +0000 UTC m=+78.254586685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wl6px" (UID: "90cc2e83-c8f6-4f31-8bdc-389922945c35") : secret "insights-runtime-extractor-tls" not found Jul 24 22:16:04.868272 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:04.868249 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/90cc2e83-c8f6-4f31-8bdc-389922945c35-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:16:07.889969 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.889937 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:16:07.889969 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.889972 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:16:07.892791 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.892774 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 22:16:07.893583 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.893567 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 22:16:07.902894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.902871 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3429398f-e14d-41cc-aa66-70b22fa7e963-metrics-certs\") pod \"network-metrics-daemon-kpxxn\" (UID: \"3429398f-e14d-41cc-aa66-70b22fa7e963\") " pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:16:07.903005 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.902918 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/259c7f52-5c99-47cf-bd40-9aea3ea0fc6d-original-pull-secret\") pod \"global-pull-secret-syncer-xvlrl\" (UID: \"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d\") " pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:16:07.990730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.990707 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:16:07.993636 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:07.993619 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 22:16:08.004117 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.004091 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 22:16:08.014519 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.014492 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7gl8\" (UniqueName: \"kubernetes.io/projected/d9f0a9b5-1255-470c-94a6-8fc07f32e6a0-kube-api-access-m7gl8\") pod \"network-check-target-bw5sl\" (UID: \"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0\") " pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:16:08.192002 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.191947 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rtdx2\"" Jul 24 22:16:08.196140 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.196124 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xvlrl" Jul 24 22:16:08.199935 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.199921 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:16:08.206467 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.206446 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-767d7\"" Jul 24 22:16:08.214049 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.214029 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kpxxn" Jul 24 22:16:08.347752 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.347720 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xvlrl"] Jul 24 22:16:08.350566 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:08.350539 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259c7f52_5c99_47cf_bd40_9aea3ea0fc6d.slice/crio-cd0a03228b8eabc57cbe4ccd13689b1b345f61ebaddcae7a1ff86c6ef56f8f78 WatchSource:0}: Error finding container cd0a03228b8eabc57cbe4ccd13689b1b345f61ebaddcae7a1ff86c6ef56f8f78: Status 404 returned error can't find the container with id cd0a03228b8eabc57cbe4ccd13689b1b345f61ebaddcae7a1ff86c6ef56f8f78 Jul 24 22:16:08.350949 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.350925 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bw5sl"] Jul 24 22:16:08.353560 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:08.353535 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f0a9b5_1255_470c_94a6_8fc07f32e6a0.slice/crio-cc76513d7190ed417a4202a8631c09d35cca228de82abf526b561e753c677e45 WatchSource:0}: Error finding container cc76513d7190ed417a4202a8631c09d35cca228de82abf526b561e753c677e45: Status 404 returned error can't find the container with id cc76513d7190ed417a4202a8631c09d35cca228de82abf526b561e753c677e45 Jul 24 22:16:08.367226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.367184 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kpxxn"] Jul 24 22:16:08.369349 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:08.369331 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3429398f_e14d_41cc_aa66_70b22fa7e963.slice/crio-dd2f48c76cf1167ba24686536f7449c2f869c1fc98437cbb0e0a51282cbdba3a WatchSource:0}: Error finding container dd2f48c76cf1167ba24686536f7449c2f869c1fc98437cbb0e0a51282cbdba3a: Status 404 returned error can't find the container with id dd2f48c76cf1167ba24686536f7449c2f869c1fc98437cbb0e0a51282cbdba3a Jul 24 22:16:08.451511 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.451458 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kpxxn" event={"ID":"3429398f-e14d-41cc-aa66-70b22fa7e963","Type":"ContainerStarted","Data":"dd2f48c76cf1167ba24686536f7449c2f869c1fc98437cbb0e0a51282cbdba3a"} Jul 24 22:16:08.452628 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.452606 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bw5sl" event={"ID":"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0","Type":"ContainerStarted","Data":"cc76513d7190ed417a4202a8631c09d35cca228de82abf526b561e753c677e45"} Jul 24 22:16:08.453618 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:08.453596 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xvlrl" event={"ID":"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d","Type":"ContainerStarted","Data":"cd0a03228b8eabc57cbe4ccd13689b1b345f61ebaddcae7a1ff86c6ef56f8f78"} Jul 24 22:16:09.805196 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:09.805110 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:16:09.807699 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:09.807673 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bd4a67d-4043-43f8-bdfd-cdb12975a5e8-metrics-tls\") pod \"dns-default-7ln87\" (UID: \"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8\") " pod="openshift-dns/dns-default-7ln87" Jul 24 22:16:10.070243 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.070166 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wnkgk\"" Jul 24 22:16:10.077863 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.077831 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7ln87" Jul 24 22:16:10.230727 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.230698 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7ln87"] Jul 24 22:16:10.233809 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:10.233781 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd4a67d_4043_43f8_bdfd_cdb12975a5e8.slice/crio-bb8461f4715698ac50df9d24f5b59d3dbcf82074ab21ba4075588e4491fd0d4f WatchSource:0}: Error finding container bb8461f4715698ac50df9d24f5b59d3dbcf82074ab21ba4075588e4491fd0d4f: Status 404 returned error can't find the container with id bb8461f4715698ac50df9d24f5b59d3dbcf82074ab21ba4075588e4491fd0d4f Jul 24 22:16:10.461388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.461359 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7ln87" event={"ID":"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8","Type":"ContainerStarted","Data":"bb8461f4715698ac50df9d24f5b59d3dbcf82074ab21ba4075588e4491fd0d4f"} Jul 24 22:16:10.463332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.463304 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kpxxn" event={"ID":"3429398f-e14d-41cc-aa66-70b22fa7e963","Type":"ContainerStarted","Data":"b29a19499498f920ebd399bb4d796a5ed6838b34fb22fc1d7eccc72a3f306b25"} Jul 24 22:16:10.463455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.463356 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kpxxn" event={"ID":"3429398f-e14d-41cc-aa66-70b22fa7e963","Type":"ContainerStarted","Data":"8076d979a874cd48fa9755151d9d19f508cf8aaedb91c848da2ec7e24a416cc1"} Jul 24 22:16:10.482268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:10.482222 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kpxxn" podStartSLOduration=65.453051436 podStartE2EDuration="1m6.482206601s" totalImagesPullingTime="1.029155165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:10.481067123 +0000 UTC m=+67.867885196" watchObservedRunningTime="2026-07-24 22:16:10.482206601 +0000 UTC m=+67.869024646" Jul 24 22:16:14.476672 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.476634 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bw5sl" event={"ID":"d9f0a9b5-1255-470c-94a6-8fc07f32e6a0","Type":"ContainerStarted","Data":"fa953a1d595a2230521196fc94e401db3ea24acf00bd7c79713597e53da42079"} Jul 24 22:16:14.477114 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.476781 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:16:14.478091 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.478068 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xvlrl" event={"ID":"259c7f52-5c99-47cf-bd40-9aea3ea0fc6d","Type":"ContainerStarted","Data":"f5e3a77ed8b0ca8e109888a24b7c5d4b0970ed2a08e9eb9ae0c1ccdf8e665c46"} Jul 24 22:16:14.479700 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.479677 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7ln87" event={"ID":"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8","Type":"ContainerStarted","Data":"1c3beff801a9d17a8030734eb925d821d156bc1262b95372974872b9f7685fec"} Jul 24 22:16:14.479781 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.479723 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7ln87" event={"ID":"3bd4a67d-4043-43f8-bdfd-cdb12975a5e8","Type":"ContainerStarted","Data":"6330d82bdacebc3df95f209f27d9d6e6babbd8eab029d7ca34e3b3406cb0f5ad"} Jul 24 22:16:14.479781 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.479747 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7ln87" Jul 24 22:16:14.514477 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.514440 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bw5sl" podStartSLOduration=65.303375561 podStartE2EDuration="1m10.514425869s" totalImagesPullingTime="5.211050308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:14.51300528 +0000 UTC m=+71.899823336" watchObservedRunningTime="2026-07-24 22:16:14.514425869 +0000 UTC m=+71.901243912" Jul 24 22:16:14.535772 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.535728 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xvlrl" podStartSLOduration=65.298693325 podStartE2EDuration="1m10.535712186s" totalImagesPullingTime="5.237018861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:14.535713818 +0000 UTC m=+71.922531864" watchObservedRunningTime="2026-07-24 22:16:14.535712186 +0000 UTC m=+71.922530227" Jul 24 22:16:14.554015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:14.553977 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7ln87" podStartSLOduration=34.196453097 podStartE2EDuration="37.553966318s" totalImagesPullingTime="3.357513221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:14.553100463 +0000 UTC m=+71.939918518" watchObservedRunningTime="2026-07-24 22:16:14.553966318 +0000 UTC m=+71.940784361" Jul 24 22:16:16.323621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.323594 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh"] Jul 24 22:16:16.326065 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.326051 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.330708 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.330683 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 22:16:16.330837 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.330728 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 24 22:16:16.332145 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.332129 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 24 22:16:16.332242 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.332194 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 22:16:16.332242 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.332222 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 22:16:16.332242 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.332209 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 24 22:16:16.332395 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.332136 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 24 22:16:16.344893 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.344873 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh"] Jul 24 22:16:16.449869 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.449851 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.449971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.449896 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-ca\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.450027 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.449980 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq26n\" (UniqueName: \"kubernetes.io/projected/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-kube-api-access-lq26n\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.450027 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.450007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-hub\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.450140 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.450036 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.450140 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.450099 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.550842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.550819 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-ca\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.550940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.550863 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq26n\" (UniqueName: \"kubernetes.io/projected/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-kube-api-access-lq26n\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.550940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.550881 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-hub\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.550940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.550901 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.550940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.550933 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.551134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.550949 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.551580 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.551556 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.553412 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.553390 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-hub\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.553601 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.553572 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.553849 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.553832 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.553912 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.553894 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-ca\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.559683 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.559662 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq26n\" (UniqueName: \"kubernetes.io/projected/0d02d2ea-9424-4dbb-91b8-264d7d28cdeb-kube-api-access-lq26n\") pod \"cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh\" (UID: \"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.655325 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.655305 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" Jul 24 22:16:16.779934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:16.779912 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh"] Jul 24 22:16:16.782079 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:16.782050 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d02d2ea_9424_4dbb_91b8_264d7d28cdeb.slice/crio-4c7e2d39eb228c8428663cef46556c2eb2e038ba632ee58f8d83d5f4e054088d WatchSource:0}: Error finding container 4c7e2d39eb228c8428663cef46556c2eb2e038ba632ee58f8d83d5f4e054088d: Status 404 returned error can't find the container with id 4c7e2d39eb228c8428663cef46556c2eb2e038ba632ee58f8d83d5f4e054088d Jul 24 22:16:17.490033 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.490007 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" event={"ID":"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb","Type":"ContainerStarted","Data":"4c7e2d39eb228c8428663cef46556c2eb2e038ba632ee58f8d83d5f4e054088d"} Jul 24 22:16:17.558504 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.558478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:16:17.558594 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.558553 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:17.559139 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.559115 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c537e5-5787-4685-8933-340514153061-service-ca-bundle\") pod \"router-default-858d756bd9-xqm8h\" (UID: \"43c537e5-5787-4685-8933-340514153061\") " pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:17.560892 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.560875 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ff0d475-2d6b-4854-8748-93f562053c6f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-b5ck4\" (UID: \"2ff0d475-2d6b-4854-8748-93f562053c6f\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:16:17.729736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.729710 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tkms8\"" Jul 24 22:16:17.738053 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.737895 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:17.797592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.797367 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xxpnc\"" Jul 24 22:16:17.806873 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.806553 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" Jul 24 22:16:17.891824 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.891795 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-858d756bd9-xqm8h"] Jul 24 22:16:17.896662 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:17.896630 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c537e5_5787_4685_8933_340514153061.slice/crio-6053253eae7cdc452821fafd6b047da91287b8a9361f581fd0229285d686201c WatchSource:0}: Error finding container 6053253eae7cdc452821fafd6b047da91287b8a9361f581fd0229285d686201c: Status 404 returned error can't find the container with id 6053253eae7cdc452821fafd6b047da91287b8a9361f581fd0229285d686201c Jul 24 22:16:17.978567 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:17.978504 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4"] Jul 24 22:16:17.982911 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:17.982880 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff0d475_2d6b_4854_8748_93f562053c6f.slice/crio-9f9f8a336d3b9e29c9af961e8152fbd7dac50a26438ab572dfc3f05a892bf521 WatchSource:0}: Error finding container 9f9f8a336d3b9e29c9af961e8152fbd7dac50a26438ab572dfc3f05a892bf521: Status 404 returned error can't find the container with id 9f9f8a336d3b9e29c9af961e8152fbd7dac50a26438ab572dfc3f05a892bf521 Jul 24 22:16:18.496495 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:18.496371 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" event={"ID":"2ff0d475-2d6b-4854-8748-93f562053c6f","Type":"ContainerStarted","Data":"9f9f8a336d3b9e29c9af961e8152fbd7dac50a26438ab572dfc3f05a892bf521"} Jul 24 22:16:18.498416 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:18.498388 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-858d756bd9-xqm8h" event={"ID":"43c537e5-5787-4685-8933-340514153061","Type":"ContainerStarted","Data":"c4753a16570817aadffc10dcff4d62de3230039fd22636e73584043c27a57a6b"} Jul 24 22:16:18.498554 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:18.498439 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-858d756bd9-xqm8h" event={"ID":"43c537e5-5787-4685-8933-340514153061","Type":"ContainerStarted","Data":"6053253eae7cdc452821fafd6b047da91287b8a9361f581fd0229285d686201c"} Jul 24 22:16:18.520825 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:18.520774 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-858d756bd9-xqm8h" podStartSLOduration=33.520757988 podStartE2EDuration="33.520757988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:18.520320054 +0000 UTC m=+75.907138099" watchObservedRunningTime="2026-07-24 22:16:18.520757988 +0000 UTC m=+75.907576035" Jul 24 22:16:18.738650 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:18.738572 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:18.741742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:18.741707 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:19.502931 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:19.502898 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:19.504272 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:19.504248 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-858d756bd9-xqm8h" Jul 24 22:16:20.887394 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:20.887360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:16:20.889964 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:20.889942 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/90cc2e83-c8f6-4f31-8bdc-389922945c35-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wl6px\" (UID: \"90cc2e83-c8f6-4f31-8bdc-389922945c35\") " pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:16:20.899023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:20.898999 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5nftg\"" Jul 24 22:16:20.906455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:20.906431 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wl6px" Jul 24 22:16:21.196002 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.195931 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wl6px"] Jul 24 22:16:21.209131 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:21.209106 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90cc2e83_c8f6_4f31_8bdc_389922945c35.slice/crio-e5998cf2da766440a3797c326be84891f6f44d14ca43c3210b2a2a9c22dcbf19 WatchSource:0}: Error finding container e5998cf2da766440a3797c326be84891f6f44d14ca43c3210b2a2a9c22dcbf19: Status 404 returned error can't find the container with id e5998cf2da766440a3797c326be84891f6f44d14ca43c3210b2a2a9c22dcbf19 Jul 24 22:16:21.511416 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.511343 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wl6px" event={"ID":"90cc2e83-c8f6-4f31-8bdc-389922945c35","Type":"ContainerStarted","Data":"e5998cf2da766440a3797c326be84891f6f44d14ca43c3210b2a2a9c22dcbf19"} Jul 24 22:16:21.514121 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.514091 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" event={"ID":"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb","Type":"ContainerStarted","Data":"7619fe430e536e2d7423f1d6946df78134a45df8c34600b20bd916f3ab7bb7c6"} Jul 24 22:16:21.514277 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.514138 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" event={"ID":"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb","Type":"ContainerStarted","Data":"4b5a7ed7905939d0c587a6e5256c8a43550be9e9c95ade62ab4454202a9a7f8c"} Jul 24 22:16:21.514277 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.514174 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" event={"ID":"0d02d2ea-9424-4dbb-91b8-264d7d28cdeb","Type":"ContainerStarted","Data":"ac08b61aeca0244e9eb8a5110751fb64f770b8e1f69ed2af1938d03cf5e9dd50"} Jul 24 22:16:21.516238 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.516210 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" event={"ID":"2ff0d475-2d6b-4854-8748-93f562053c6f","Type":"ContainerStarted","Data":"9cf18ca7069a6d2a8d621866bed440d35ee4e9d87138f9a933beab88bb7e5007"} Jul 24 22:16:21.536769 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.536730 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5cc6fcfbd6-dnwwh" podStartSLOduration=1.237530994 podStartE2EDuration="5.536714323s" totalImagesPullingTime="4.299183329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:21.535345579 +0000 UTC m=+78.922163644" watchObservedRunningTime="2026-07-24 22:16:21.536714323 +0000 UTC m=+78.923532347" Jul 24 22:16:21.553875 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:21.553837 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-b5ck4" podStartSLOduration=33.467874573 podStartE2EDuration="36.553826039s" totalImagesPullingTime="3.085951466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:21.552852585 +0000 UTC m=+78.939670653" watchObservedRunningTime="2026-07-24 22:16:21.553826039 +0000 UTC m=+78.940644082" Jul 24 22:16:24.485510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:24.485403 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7ln87" Jul 24 22:16:26.536754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:26.536715 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wl6px" event={"ID":"90cc2e83-c8f6-4f31-8bdc-389922945c35","Type":"ContainerStarted","Data":"8a46cfa85ee3803f767438fa9cca452cdb57f10f5dc59f10ec751e76a559abad"} Jul 24 22:16:29.549096 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:29.549070 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wl6px" event={"ID":"90cc2e83-c8f6-4f31-8bdc-389922945c35","Type":"ContainerStarted","Data":"9e693ef349bc73683c5e705cf9b789a3bdae4451cd1ce0ebd71f7d5b2f47964a"} Jul 24 22:16:33.032833 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.032803 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-86l6w"] Jul 24 22:16:33.034794 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.034780 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.038051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.038024 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 22:16:33.038306 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.038288 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 22:16:33.038538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.038524 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gv6z9\"" Jul 24 22:16:33.038937 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.038925 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 22:16:33.039429 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.039418 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 22:16:33.178738 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178712 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-textfile\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178756 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-tls\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178803 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d876e8d-ee84-4be9-8204-d9c8431caea5-metrics-client-ca\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178832 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w244\" (UniqueName: \"kubernetes.io/projected/8d876e8d-ee84-4be9-8204-d9c8431caea5-kube-api-access-5w244\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178854 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-root\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178883 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-wtmp\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178905 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-sys\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.178933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178921 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.179061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.178967 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-accelerators-collector-config\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279331 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279309 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-accelerators-collector-config\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279343 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-textfile\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279359 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-tls\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279376 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d876e8d-ee84-4be9-8204-d9c8431caea5-metrics-client-ca\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279393 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w244\" (UniqueName: \"kubernetes.io/projected/8d876e8d-ee84-4be9-8204-d9c8431caea5-kube-api-access-5w244\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279409 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-root\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279465 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-root\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279497 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-wtmp\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:33.279520 2554 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279544 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-sys\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:16:33.279599 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-tls podName:8d876e8d-ee84-4be9-8204-d9c8431caea5 nodeName:}" failed. No retries permitted until 2026-07-24 22:16:33.779579272 +0000 UTC m=+91.166397296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-tls") pod "node-exporter-86l6w" (UID: "8d876e8d-ee84-4be9-8204-d9c8431caea5") : secret "node-exporter-tls" not found Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279626 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279669 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-wtmp\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279681 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d876e8d-ee84-4be9-8204-d9c8431caea5-sys\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.279779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279689 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-textfile\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.280129 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.279948 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-accelerators-collector-config\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.280129 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.280032 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d876e8d-ee84-4be9-8204-d9c8431caea5-metrics-client-ca\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.281892 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.281871 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.297335 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.297281 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w244\" (UniqueName: \"kubernetes.io/projected/8d876e8d-ee84-4be9-8204-d9c8431caea5-kube-api-access-5w244\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.567457 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.567387 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wl6px" event={"ID":"90cc2e83-c8f6-4f31-8bdc-389922945c35","Type":"ContainerStarted","Data":"e302096b75834723f346ce2a1399f5e36dd85889dc929a90ae23bb5e3c98724c"} Jul 24 22:16:33.586710 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.586671 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wl6px" podStartSLOduration=33.395790757 podStartE2EDuration="44.586657827s" totalImagesPullingTime="11.19086707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 22:15:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:33.585906746 +0000 UTC m=+90.972724791" watchObservedRunningTime="2026-07-24 22:16:33.586657827 +0000 UTC m=+90.973475872" Jul 24 22:16:33.784418 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.784387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-tls\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.786551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.786533 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8d876e8d-ee84-4be9-8204-d9c8431caea5-node-exporter-tls\") pod \"node-exporter-86l6w\" (UID: \"8d876e8d-ee84-4be9-8204-d9c8431caea5\") " pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.943777 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:33.943753 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-86l6w" Jul 24 22:16:33.954639 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:33.954614 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d876e8d_ee84_4be9_8204_d9c8431caea5.slice/crio-5bc074a4c4b995d9dc564184307fff6b34ffa4b98b91e8f3a5a4f1d94a7cef62 WatchSource:0}: Error finding container 5bc074a4c4b995d9dc564184307fff6b34ffa4b98b91e8f3a5a4f1d94a7cef62: Status 404 returned error can't find the container with id 5bc074a4c4b995d9dc564184307fff6b34ffa4b98b91e8f3a5a4f1d94a7cef62 Jul 24 22:16:34.572537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:34.572503 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86l6w" event={"ID":"8d876e8d-ee84-4be9-8204-d9c8431caea5","Type":"ContainerStarted","Data":"5bc074a4c4b995d9dc564184307fff6b34ffa4b98b91e8f3a5a4f1d94a7cef62"} Jul 24 22:16:35.577636 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:35.577602 2554 generic.go:356] "Generic (PLEG): container finished" podID="8d876e8d-ee84-4be9-8204-d9c8431caea5" containerID="3aa919a99fc757c3d4f9dc881b0e2b8ef57dc6915db6ce89ab9abd88a9d2d24d" exitCode=0 Jul 24 22:16:35.578039 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:35.577695 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86l6w" event={"ID":"8d876e8d-ee84-4be9-8204-d9c8431caea5","Type":"ContainerDied","Data":"3aa919a99fc757c3d4f9dc881b0e2b8ef57dc6915db6ce89ab9abd88a9d2d24d"} Jul 24 22:16:36.583543 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:36.583511 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86l6w" event={"ID":"8d876e8d-ee84-4be9-8204-d9c8431caea5","Type":"ContainerStarted","Data":"5e2ceed76ba73d5bf6b35d849be408acdc5b40861eda49a8d6412829b9b99058"} Jul 24 22:16:36.584002 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:36.583561 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-86l6w" event={"ID":"8d876e8d-ee84-4be9-8204-d9c8431caea5","Type":"ContainerStarted","Data":"c7f883ac506b4310f0560d20e51cfb1b7b33756a9aa868999fd496af3e7eab8c"} Jul 24 22:16:36.616475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:36.616431 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-86l6w" podStartSLOduration=2.5459391289999997 podStartE2EDuration="3.616415786s" totalImagesPullingTime="1.002223244s" totalInitContainerRuntime="68.253413ms" isStatefulPod=true podCreationTimestamp="2026-07-24 22:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:36.614885464 +0000 UTC m=+94.001703527" watchObservedRunningTime="2026-07-24 22:16:36.616415786 +0000 UTC m=+94.003233830" Jul 24 22:16:39.915487 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.915454 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cd6b7dbb-qbbn7"] Jul 24 22:16:39.918585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.918567 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:39.921850 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.921829 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 22:16:39.922001 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.921987 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 22:16:39.923111 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.923067 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 22:16:39.923111 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.923102 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 22:16:39.925949 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.925917 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 22:16:39.926048 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.925991 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 22:16:39.926194 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.926179 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-shd2w\"" Jul 24 22:16:39.926280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.926234 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 22:16:39.931574 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:39.931554 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cd6b7dbb-qbbn7"] Jul 24 22:16:40.024905 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.024880 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-service-ca\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.025006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.024911 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-oauth-config\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.025006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.024941 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-console-config\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.025111 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.025004 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-serving-cert\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.025111 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.025035 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrssn\" (UniqueName: \"kubernetes.io/projected/f50aa915-7065-44d6-84d7-59b679747c59-kube-api-access-nrssn\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.025111 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.025064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-oauth-serving-cert\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.125462 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.125440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-oauth-serving-cert\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.125552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.125478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-service-ca\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.125552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.125496 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-oauth-config\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.125552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.125515 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-console-config\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.125552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.125536 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-serving-cert\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.125748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.125663 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrssn\" (UniqueName: \"kubernetes.io/projected/f50aa915-7065-44d6-84d7-59b679747c59-kube-api-access-nrssn\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.126132 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.126112 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-oauth-serving-cert\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.126230 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.126201 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-service-ca\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.126282 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.126265 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-console-config\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.128526 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.128506 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-oauth-config\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.128585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.128534 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-serving-cert\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.134181 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.134141 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrssn\" (UniqueName: \"kubernetes.io/projected/f50aa915-7065-44d6-84d7-59b679747c59-kube-api-access-nrssn\") pod \"console-7cd6b7dbb-qbbn7\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.228537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.228488 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:40.355125 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.355101 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cd6b7dbb-qbbn7"] Jul 24 22:16:40.357285 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:16:40.357258 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50aa915_7065_44d6_84d7_59b679747c59.slice/crio-177797b023fb997b6da5fdd4c1afe42e1483cc9ab2663f9e8980e4e4f3f06ec7 WatchSource:0}: Error finding container 177797b023fb997b6da5fdd4c1afe42e1483cc9ab2663f9e8980e4e4f3f06ec7: Status 404 returned error can't find the container with id 177797b023fb997b6da5fdd4c1afe42e1483cc9ab2663f9e8980e4e4f3f06ec7 Jul 24 22:16:40.599366 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:40.599299 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd6b7dbb-qbbn7" event={"ID":"f50aa915-7065-44d6-84d7-59b679747c59","Type":"ContainerStarted","Data":"177797b023fb997b6da5fdd4c1afe42e1483cc9ab2663f9e8980e4e4f3f06ec7"} Jul 24 22:16:43.611100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:43.611068 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd6b7dbb-qbbn7" event={"ID":"f50aa915-7065-44d6-84d7-59b679747c59","Type":"ContainerStarted","Data":"349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b"} Jul 24 22:16:43.628238 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:43.628196 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cd6b7dbb-qbbn7" podStartSLOduration=1.970464312 podStartE2EDuration="4.628181615s" totalImagesPullingTime="2.657717303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:16:43.627594258 +0000 UTC m=+101.014412322" watchObservedRunningTime="2026-07-24 22:16:43.628181615 +0000 UTC m=+101.014999654" Jul 24 22:16:45.485227 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:45.485198 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bw5sl" Jul 24 22:16:50.228661 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:50.228629 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:50.229054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:50.228686 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:50.233252 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:50.233232 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:50.635686 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:50.635622 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:16:54.647444 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:54.647408 2554 generic.go:356] "Generic (PLEG): container finished" podID="36fd0bd6-9a6b-4676-844c-0802bce022a5" containerID="5417ef9add8a445e5039f8c0f52cc5fd771f2c2093b4d983b0aa9c27f3d27f86" exitCode=0 Jul 24 22:16:54.647826 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:54.647480 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mm46z" event={"ID":"36fd0bd6-9a6b-4676-844c-0802bce022a5","Type":"ContainerDied","Data":"5417ef9add8a445e5039f8c0f52cc5fd771f2c2093b4d983b0aa9c27f3d27f86"} Jul 24 22:16:54.647911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:54.647896 2554 scope.go:117] "RemoveContainer" containerID="5417ef9add8a445e5039f8c0f52cc5fd771f2c2093b4d983b0aa9c27f3d27f86" Jul 24 22:16:55.652676 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:55.652642 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-mm46z" event={"ID":"36fd0bd6-9a6b-4676-844c-0802bce022a5","Type":"ContainerStarted","Data":"c80c297b692f1fbf29f1d82c5bfab2f10c286a5c166e25628f3b87eff3a9c5c0"} Jul 24 22:16:58.768866 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:16:58.768833 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cd6b7dbb-qbbn7"] Jul 24 22:17:23.789540 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:23.789475 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cd6b7dbb-qbbn7" podUID="f50aa915-7065-44d6-84d7-59b679747c59" containerName="console" containerID="cri-o://349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b" gracePeriod=15 Jul 24 22:17:24.028195 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.028175 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:17:24.036621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.036603 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-console-config\") pod \"f50aa915-7065-44d6-84d7-59b679747c59\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " Jul 24 22:17:24.036696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.036635 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrssn\" (UniqueName: \"kubernetes.io/projected/f50aa915-7065-44d6-84d7-59b679747c59-kube-api-access-nrssn\") pod \"f50aa915-7065-44d6-84d7-59b679747c59\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " Jul 24 22:17:24.036696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.036660 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-service-ca\") pod \"f50aa915-7065-44d6-84d7-59b679747c59\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " Jul 24 22:17:24.036696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.036690 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-serving-cert\") pod \"f50aa915-7065-44d6-84d7-59b679747c59\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " Jul 24 22:17:24.036844 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.036717 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-oauth-config\") pod \"f50aa915-7065-44d6-84d7-59b679747c59\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " Jul 24 22:17:24.036844 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.036733 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-oauth-serving-cert\") pod \"f50aa915-7065-44d6-84d7-59b679747c59\" (UID: \"f50aa915-7065-44d6-84d7-59b679747c59\") " Jul 24 22:17:24.037180 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.037128 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-console-config" (OuterVolumeSpecName: "console-config") pod "f50aa915-7065-44d6-84d7-59b679747c59" (UID: "f50aa915-7065-44d6-84d7-59b679747c59"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:17:24.037300 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.037264 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f50aa915-7065-44d6-84d7-59b679747c59" (UID: "f50aa915-7065-44d6-84d7-59b679747c59"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:17:24.037300 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.037258 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-service-ca" (OuterVolumeSpecName: "service-ca") pod "f50aa915-7065-44d6-84d7-59b679747c59" (UID: "f50aa915-7065-44d6-84d7-59b679747c59"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:17:24.038954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.038928 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f50aa915-7065-44d6-84d7-59b679747c59" (UID: "f50aa915-7065-44d6-84d7-59b679747c59"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:17:24.039051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.039000 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f50aa915-7065-44d6-84d7-59b679747c59" (UID: "f50aa915-7065-44d6-84d7-59b679747c59"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:17:24.039116 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.039054 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50aa915-7065-44d6-84d7-59b679747c59-kube-api-access-nrssn" (OuterVolumeSpecName: "kube-api-access-nrssn") pod "f50aa915-7065-44d6-84d7-59b679747c59" (UID: "f50aa915-7065-44d6-84d7-59b679747c59"). InnerVolumeSpecName "kube-api-access-nrssn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:17:24.137567 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.137531 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-service-ca\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:17:24.137567 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.137566 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-serving-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:17:24.137765 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.137581 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f50aa915-7065-44d6-84d7-59b679747c59-console-oauth-config\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:17:24.137765 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.137590 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-oauth-serving-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:17:24.137765 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.137598 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f50aa915-7065-44d6-84d7-59b679747c59-console-config\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:17:24.137765 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.137605 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrssn\" (UniqueName: \"kubernetes.io/projected/f50aa915-7065-44d6-84d7-59b679747c59-kube-api-access-nrssn\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:17:24.755100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.755061 2554 generic.go:356] "Generic (PLEG): container finished" podID="f50aa915-7065-44d6-84d7-59b679747c59" containerID="349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b" exitCode=0 Jul 24 22:17:24.755273 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.755121 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cd6b7dbb-qbbn7" Jul 24 22:17:24.755273 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.755141 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd6b7dbb-qbbn7" event={"ID":"f50aa915-7065-44d6-84d7-59b679747c59","Type":"ContainerDied","Data":"349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b"} Jul 24 22:17:24.755273 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.755235 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cd6b7dbb-qbbn7" event={"ID":"f50aa915-7065-44d6-84d7-59b679747c59","Type":"ContainerDied","Data":"177797b023fb997b6da5fdd4c1afe42e1483cc9ab2663f9e8980e4e4f3f06ec7"} Jul 24 22:17:24.755273 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.755254 2554 scope.go:117] "RemoveContainer" containerID="349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b" Jul 24 22:17:24.763662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.763646 2554 scope.go:117] "RemoveContainer" containerID="349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b" Jul 24 22:17:24.763913 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:17:24.763895 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b\": container with ID starting with 349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b not found: ID does not exist" containerID="349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b" Jul 24 22:17:24.763974 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.763921 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b"} err="failed to get container status \"349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b\": rpc error: code = NotFound desc = could not find container \"349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b\": container with ID starting with 349c59e74950f33a8dc2bde9ac73b52b9a4364dc40c8ade80dce0e7d6f0ea01b not found: ID does not exist" Jul 24 22:17:24.774959 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.774940 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cd6b7dbb-qbbn7"] Jul 24 22:17:24.778939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:24.778914 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cd6b7dbb-qbbn7"] Jul 24 22:17:25.175023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:17:25.174993 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50aa915-7065-44d6-84d7-59b679747c59" path="/var/lib/kubelet/pods/f50aa915-7065-44d6-84d7-59b679747c59/volumes" Jul 24 22:19:30.803303 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.803227 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78dd7d4468-klt8g"] Jul 24 22:19:30.803742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.803466 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50aa915-7065-44d6-84d7-59b679747c59" containerName="console" Jul 24 22:19:30.803742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.803477 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50aa915-7065-44d6-84d7-59b679747c59" containerName="console" Jul 24 22:19:30.803742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.803542 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50aa915-7065-44d6-84d7-59b679747c59" containerName="console" Jul 24 22:19:30.805340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.805325 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.807779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.807757 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 22:19:30.807934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.807844 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 22:19:30.807934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.807879 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 22:19:30.808057 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.807932 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 22:19:30.808970 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.808946 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 22:19:30.809214 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.809198 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 22:19:30.809284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.809261 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 22:19:30.809284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.809269 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-shd2w\"" Jul 24 22:19:30.813705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.813687 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 22:19:30.816115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.816096 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78dd7d4468-klt8g"] Jul 24 22:19:30.826283 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826264 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-serving-cert\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.826376 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826304 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-trusted-ca-bundle\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.826376 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826358 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-oauth-serving-cert\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.826455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826391 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-console-config\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.826455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826411 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4dxh\" (UniqueName: \"kubernetes.io/projected/97b4da9f-9712-4abb-b16e-51c52129f449-kube-api-access-q4dxh\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.826455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826431 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-service-ca\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.826455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.826447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-oauth-config\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.926944 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.926908 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-trusted-ca-bundle\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.926944 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.926947 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-oauth-serving-cert\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.926972 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-console-config\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.926999 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4dxh\" (UniqueName: \"kubernetes.io/projected/97b4da9f-9712-4abb-b16e-51c52129f449-kube-api-access-q4dxh\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.927026 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-service-ca\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.927048 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-oauth-config\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.927086 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-serving-cert\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927756 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.927728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-oauth-serving-cert\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.927978 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.927962 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-service-ca\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.928264 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.928242 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-trusted-ca-bundle\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.928345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.928261 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-console-config\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.929668 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.929645 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-serving-cert\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.929832 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.929818 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-oauth-config\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:30.935579 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:30.935558 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4dxh\" (UniqueName: \"kubernetes.io/projected/97b4da9f-9712-4abb-b16e-51c52129f449-kube-api-access-q4dxh\") pod \"console-78dd7d4468-klt8g\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:31.115363 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:31.115292 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:31.247405 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:31.247374 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78dd7d4468-klt8g"] Jul 24 22:19:31.250035 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:19:31.250004 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b4da9f_9712_4abb_b16e_51c52129f449.slice/crio-1a925cc851a88d2dc9a81bd293c63613bb13e716d36cfab371e625d6d9ace510 WatchSource:0}: Error finding container 1a925cc851a88d2dc9a81bd293c63613bb13e716d36cfab371e625d6d9ace510: Status 404 returned error can't find the container with id 1a925cc851a88d2dc9a81bd293c63613bb13e716d36cfab371e625d6d9ace510 Jul 24 22:19:32.170862 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:32.170823 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dd7d4468-klt8g" event={"ID":"97b4da9f-9712-4abb-b16e-51c52129f449","Type":"ContainerStarted","Data":"23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9"} Jul 24 22:19:32.170862 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:32.170869 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dd7d4468-klt8g" event={"ID":"97b4da9f-9712-4abb-b16e-51c52129f449","Type":"ContainerStarted","Data":"1a925cc851a88d2dc9a81bd293c63613bb13e716d36cfab371e625d6d9ace510"} Jul 24 22:19:32.188475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:32.188428 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78dd7d4468-klt8g" podStartSLOduration=2.188414265 podStartE2EDuration="2.188414265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:19:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:19:32.186731229 +0000 UTC m=+269.573549275" watchObservedRunningTime="2026-07-24 22:19:32.188414265 +0000 UTC m=+269.575232311" Jul 24 22:19:41.115999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:41.115954 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:41.116560 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:41.116030 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:41.120703 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:41.120677 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:19:41.205775 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:19:41.205747 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:20:00.662340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.662304 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb"] Jul 24 22:20:00.664311 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.664296 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.666814 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.666796 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 22:20:00.667838 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.667818 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g5kc7\"" Jul 24 22:20:00.667922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.667827 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 22:20:00.674629 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.674611 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb"] Jul 24 22:20:00.747539 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.747506 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h75f6\" (UniqueName: \"kubernetes.io/projected/54762e7e-67a2-4e95-8171-691628285957-kube-api-access-h75f6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.747539 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.747545 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.747841 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.747568 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.848757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.848717 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h75f6\" (UniqueName: \"kubernetes.io/projected/54762e7e-67a2-4e95-8171-691628285957-kube-api-access-h75f6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.848757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.848764 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.848998 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.848786 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.849142 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.849124 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.849210 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.849143 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.857323 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.857291 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h75f6\" (UniqueName: \"kubernetes.io/projected/54762e7e-67a2-4e95-8171-691628285957-kube-api-access-h75f6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:00.973210 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:00.973108 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:01.100609 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:01.100579 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb"] Jul 24 22:20:01.266958 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:01.266868 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" event={"ID":"54762e7e-67a2-4e95-8171-691628285957","Type":"ContainerStarted","Data":"09cdd73400d902f6ae54a6cc81641e890b9e59884e12ff1e925a4239837052e8"} Jul 24 22:20:02.992702 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:02.992671 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 22:20:08.294736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:08.294701 2554 generic.go:356] "Generic (PLEG): container finished" podID="54762e7e-67a2-4e95-8171-691628285957" containerID="575271ae788ec781879bd23c08ad4a10962170b3e601826f5163ae5b975dddcc" exitCode=0 Jul 24 22:20:08.295382 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:08.294763 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" event={"ID":"54762e7e-67a2-4e95-8171-691628285957","Type":"ContainerDied","Data":"575271ae788ec781879bd23c08ad4a10962170b3e601826f5163ae5b975dddcc"} Jul 24 22:20:08.295775 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:08.295759 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 22:20:11.309227 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:11.309179 2554 generic.go:356] "Generic (PLEG): container finished" podID="54762e7e-67a2-4e95-8171-691628285957" containerID="6bb5d1a5ad51b2bcff91d4f56657a73717825105dfe6fcefe6cae4d368661985" exitCode=0 Jul 24 22:20:11.309582 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:11.309254 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" event={"ID":"54762e7e-67a2-4e95-8171-691628285957","Type":"ContainerDied","Data":"6bb5d1a5ad51b2bcff91d4f56657a73717825105dfe6fcefe6cae4d368661985"} Jul 24 22:20:19.342462 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:19.342422 2554 generic.go:356] "Generic (PLEG): container finished" podID="54762e7e-67a2-4e95-8171-691628285957" containerID="90067a8ba6b0a8c3b3b4eabe07902584c7c56503d0ccd0c8bc6e30c9479275ba" exitCode=0 Jul 24 22:20:19.342844 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:19.342486 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" event={"ID":"54762e7e-67a2-4e95-8171-691628285957","Type":"ContainerDied","Data":"90067a8ba6b0a8c3b3b4eabe07902584c7c56503d0ccd0c8bc6e30c9479275ba"} Jul 24 22:20:20.474133 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.474106 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:20.614963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.614868 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h75f6\" (UniqueName: \"kubernetes.io/projected/54762e7e-67a2-4e95-8171-691628285957-kube-api-access-h75f6\") pod \"54762e7e-67a2-4e95-8171-691628285957\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " Jul 24 22:20:20.614963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.614931 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-util\") pod \"54762e7e-67a2-4e95-8171-691628285957\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " Jul 24 22:20:20.614963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.614961 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-bundle\") pod \"54762e7e-67a2-4e95-8171-691628285957\" (UID: \"54762e7e-67a2-4e95-8171-691628285957\") " Jul 24 22:20:20.615649 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.615618 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-bundle" (OuterVolumeSpecName: "bundle") pod "54762e7e-67a2-4e95-8171-691628285957" (UID: "54762e7e-67a2-4e95-8171-691628285957"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:20:20.617320 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.617292 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54762e7e-67a2-4e95-8171-691628285957-kube-api-access-h75f6" (OuterVolumeSpecName: "kube-api-access-h75f6") pod "54762e7e-67a2-4e95-8171-691628285957" (UID: "54762e7e-67a2-4e95-8171-691628285957"). InnerVolumeSpecName "kube-api-access-h75f6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:20:20.618952 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.618923 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-util" (OuterVolumeSpecName: "util") pod "54762e7e-67a2-4e95-8171-691628285957" (UID: "54762e7e-67a2-4e95-8171-691628285957"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:20:20.715982 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.715934 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h75f6\" (UniqueName: \"kubernetes.io/projected/54762e7e-67a2-4e95-8171-691628285957-kube-api-access-h75f6\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:20.715982 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.715978 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:20.715982 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:20.715987 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54762e7e-67a2-4e95-8171-691628285957-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:21.352639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:21.352601 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" event={"ID":"54762e7e-67a2-4e95-8171-691628285957","Type":"ContainerDied","Data":"09cdd73400d902f6ae54a6cc81641e890b9e59884e12ff1e925a4239837052e8"} Jul 24 22:20:21.352639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:21.352631 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q9qjb" Jul 24 22:20:21.352817 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:21.352658 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="09cdd73400d902f6ae54a6cc81641e890b9e59884e12ff1e925a4239837052e8" Jul 24 22:20:33.235127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235085 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw"] Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235355 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="util" Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235367 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="util" Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235375 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="pull" Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235380 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="pull" Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235395 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="extract" Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235401 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="extract" Jul 24 22:20:33.235537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.235443 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="54762e7e-67a2-4e95-8171-691628285957" containerName="extract" Jul 24 22:20:33.239840 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.239821 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.242655 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.242636 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g5kc7\"" Jul 24 22:20:33.243079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.243064 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 22:20:33.243784 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.243765 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 22:20:33.257831 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.257810 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw"] Jul 24 22:20:33.418170 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.418125 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww4p8\" (UniqueName: \"kubernetes.io/projected/e6f4f38b-a260-4f19-98a3-dea66710dd87-kube-api-access-ww4p8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.418275 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.418179 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.418275 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.418200 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.519304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.519242 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.519412 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.519325 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww4p8\" (UniqueName: \"kubernetes.io/projected/e6f4f38b-a260-4f19-98a3-dea66710dd87-kube-api-access-ww4p8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.519412 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.519369 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.519598 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.519570 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.519715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.519693 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.528806 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.528785 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww4p8\" (UniqueName: \"kubernetes.io/projected/e6f4f38b-a260-4f19-98a3-dea66710dd87-kube-api-access-ww4p8\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.548950 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.548930 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:33.670490 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:33.670467 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw"] Jul 24 22:20:33.672886 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:20:33.672859 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f4f38b_a260_4f19_98a3_dea66710dd87.slice/crio-9796acf34dd0082d37aec4ef3fca13fd9cd7b79e917fc0d0e922a19074a3f61c WatchSource:0}: Error finding container 9796acf34dd0082d37aec4ef3fca13fd9cd7b79e917fc0d0e922a19074a3f61c: Status 404 returned error can't find the container with id 9796acf34dd0082d37aec4ef3fca13fd9cd7b79e917fc0d0e922a19074a3f61c Jul 24 22:20:34.404851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:34.404812 2554 generic.go:356] "Generic (PLEG): container finished" podID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerID="1ff0d830069914be9325c530448358e974532ed370e34dd7323ecca1ac65b485" exitCode=0 Jul 24 22:20:34.405464 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:34.404894 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" event={"ID":"e6f4f38b-a260-4f19-98a3-dea66710dd87","Type":"ContainerDied","Data":"1ff0d830069914be9325c530448358e974532ed370e34dd7323ecca1ac65b485"} Jul 24 22:20:34.405464 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:34.404969 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" event={"ID":"e6f4f38b-a260-4f19-98a3-dea66710dd87","Type":"ContainerStarted","Data":"9796acf34dd0082d37aec4ef3fca13fd9cd7b79e917fc0d0e922a19074a3f61c"} Jul 24 22:20:37.417997 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:37.417961 2554 generic.go:356] "Generic (PLEG): container finished" podID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerID="27295ef8ccd42f3d234699d15fef9afbc68b1a334a26ffe7c0df9f21daf5bd32" exitCode=0 Jul 24 22:20:37.418414 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:37.418031 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" event={"ID":"e6f4f38b-a260-4f19-98a3-dea66710dd87","Type":"ContainerDied","Data":"27295ef8ccd42f3d234699d15fef9afbc68b1a334a26ffe7c0df9f21daf5bd32"} Jul 24 22:20:38.424618 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:38.424587 2554 generic.go:356] "Generic (PLEG): container finished" podID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerID="963cccec4d05940492039dd093b2e33f9aa1f902419004c55f444ee6a96637a0" exitCode=0 Jul 24 22:20:38.425037 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:38.424678 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" event={"ID":"e6f4f38b-a260-4f19-98a3-dea66710dd87","Type":"ContainerDied","Data":"963cccec4d05940492039dd093b2e33f9aa1f902419004c55f444ee6a96637a0"} Jul 24 22:20:39.561646 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.561622 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:39.662567 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.662529 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww4p8\" (UniqueName: \"kubernetes.io/projected/e6f4f38b-a260-4f19-98a3-dea66710dd87-kube-api-access-ww4p8\") pod \"e6f4f38b-a260-4f19-98a3-dea66710dd87\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " Jul 24 22:20:39.662567 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.662567 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-bundle\") pod \"e6f4f38b-a260-4f19-98a3-dea66710dd87\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " Jul 24 22:20:39.662773 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.662594 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-util\") pod \"e6f4f38b-a260-4f19-98a3-dea66710dd87\" (UID: \"e6f4f38b-a260-4f19-98a3-dea66710dd87\") " Jul 24 22:20:39.663036 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.663001 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-bundle" (OuterVolumeSpecName: "bundle") pod "e6f4f38b-a260-4f19-98a3-dea66710dd87" (UID: "e6f4f38b-a260-4f19-98a3-dea66710dd87"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:20:39.664989 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.664960 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f4f38b-a260-4f19-98a3-dea66710dd87-kube-api-access-ww4p8" (OuterVolumeSpecName: "kube-api-access-ww4p8") pod "e6f4f38b-a260-4f19-98a3-dea66710dd87" (UID: "e6f4f38b-a260-4f19-98a3-dea66710dd87"). InnerVolumeSpecName "kube-api-access-ww4p8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:20:39.667628 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.667603 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-util" (OuterVolumeSpecName: "util") pod "e6f4f38b-a260-4f19-98a3-dea66710dd87" (UID: "e6f4f38b-a260-4f19-98a3-dea66710dd87"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:20:39.763735 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.763645 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww4p8\" (UniqueName: \"kubernetes.io/projected/e6f4f38b-a260-4f19-98a3-dea66710dd87-kube-api-access-ww4p8\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:39.763735 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.763680 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:39.763735 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:39.763691 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e6f4f38b-a260-4f19-98a3-dea66710dd87-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:40.434862 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:40.434826 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" event={"ID":"e6f4f38b-a260-4f19-98a3-dea66710dd87","Type":"ContainerDied","Data":"9796acf34dd0082d37aec4ef3fca13fd9cd7b79e917fc0d0e922a19074a3f61c"} Jul 24 22:20:40.435004 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:40.434878 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9796acf34dd0082d37aec4ef3fca13fd9cd7b79e917fc0d0e922a19074a3f61c" Jul 24 22:20:40.435004 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:40.434836 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdqpzw" Jul 24 22:20:46.692579 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692547 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl"] Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692791 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="util" Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692801 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="util" Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692814 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="pull" Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692820 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="pull" Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692827 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="extract" Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692834 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="extract" Jul 24 22:20:46.692954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.692876 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f4f38b-a260-4f19-98a3-dea66710dd87" containerName="extract" Jul 24 22:20:46.694719 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.694704 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:46.697450 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.697427 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 22:20:46.697591 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.697471 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 22:20:46.698455 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.698438 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-f4glz\"" Jul 24 22:20:46.703783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.703761 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl"] Jul 24 22:20:46.712032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.712009 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-85swl\" (UID: \"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:46.712121 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.712083 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-867st\" (UniqueName: \"kubernetes.io/projected/34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38-kube-api-access-867st\") pod \"openshift-lws-operator-fd8ccff4c-85swl\" (UID: \"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:46.812545 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.812509 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-867st\" (UniqueName: \"kubernetes.io/projected/34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38-kube-api-access-867st\") pod \"openshift-lws-operator-fd8ccff4c-85swl\" (UID: \"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:46.812756 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.812571 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-85swl\" (UID: \"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:46.812978 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.812955 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-85swl\" (UID: \"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:46.821033 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:46.821011 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-867st\" (UniqueName: \"kubernetes.io/projected/34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38-kube-api-access-867st\") pod \"openshift-lws-operator-fd8ccff4c-85swl\" (UID: \"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:47.004421 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:47.004322 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" Jul 24 22:20:47.132035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:47.132004 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl"] Jul 24 22:20:47.135599 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:20:47.135573 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34a5c9c8_0c14_4d6a_a8a4_a5cc3a333c38.slice/crio-b41062fe15fd40b8a87a8859f84420ebd5dd02d70f2a02195edd9f16fbf7762a WatchSource:0}: Error finding container b41062fe15fd40b8a87a8859f84420ebd5dd02d70f2a02195edd9f16fbf7762a: Status 404 returned error can't find the container with id b41062fe15fd40b8a87a8859f84420ebd5dd02d70f2a02195edd9f16fbf7762a Jul 24 22:20:47.460739 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:47.460704 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" event={"ID":"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38","Type":"ContainerStarted","Data":"b41062fe15fd40b8a87a8859f84420ebd5dd02d70f2a02195edd9f16fbf7762a"} Jul 24 22:20:50.475049 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:50.475013 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" event={"ID":"34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38","Type":"ContainerStarted","Data":"3400aaca7687a39eaf6897e8efbdd1bb116fe3f94432b76e528847f409054215"} Jul 24 22:20:50.490961 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:50.490911 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-85swl" podStartSLOduration=1.635228024 podStartE2EDuration="4.490891897s" totalImagesPullingTime="2.855663873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:20:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:20:50.490207519 +0000 UTC m=+347.877025566" watchObservedRunningTime="2026-07-24 22:20:50.490891897 +0000 UTC m=+347.877709944" Jul 24 22:20:52.786401 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.786362 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k"] Jul 24 22:20:52.788551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.788536 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.791174 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.791137 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 22:20:52.792275 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.792250 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g5kc7\"" Jul 24 22:20:52.792384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.792291 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 22:20:52.799202 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.799183 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k"] Jul 24 22:20:52.859426 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.859392 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.859604 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.859436 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpx26\" (UniqueName: \"kubernetes.io/projected/6c56aa46-5936-400e-8c57-d2fb530392b1-kube-api-access-fpx26\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.859604 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.859481 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.960388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.960351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.960552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.960396 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpx26\" (UniqueName: \"kubernetes.io/projected/6c56aa46-5936-400e-8c57-d2fb530392b1-kube-api-access-fpx26\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.960552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.960440 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.960749 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.960730 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.960793 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.960780 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:52.968331 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:52.968310 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpx26\" (UniqueName: \"kubernetes.io/projected/6c56aa46-5936-400e-8c57-d2fb530392b1-kube-api-access-fpx26\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:53.097730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:53.097603 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:53.225226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:53.225176 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k"] Jul 24 22:20:53.227717 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:20:53.227690 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c56aa46_5936_400e_8c57_d2fb530392b1.slice/crio-788ab9a67a85d88fb497a7202fde733dd7ca826ba372fba9fc034ac53ac264f7 WatchSource:0}: Error finding container 788ab9a67a85d88fb497a7202fde733dd7ca826ba372fba9fc034ac53ac264f7: Status 404 returned error can't find the container with id 788ab9a67a85d88fb497a7202fde733dd7ca826ba372fba9fc034ac53ac264f7 Jul 24 22:20:53.487264 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:53.487225 2554 generic.go:356] "Generic (PLEG): container finished" podID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerID="be34cd9105fad17f1d5f453aa0e7be870f24aa22cbe11abc808905db4d1ff138" exitCode=0 Jul 24 22:20:53.487469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:53.487317 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" event={"ID":"6c56aa46-5936-400e-8c57-d2fb530392b1","Type":"ContainerDied","Data":"be34cd9105fad17f1d5f453aa0e7be870f24aa22cbe11abc808905db4d1ff138"} Jul 24 22:20:53.487469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:53.487369 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" event={"ID":"6c56aa46-5936-400e-8c57-d2fb530392b1","Type":"ContainerStarted","Data":"788ab9a67a85d88fb497a7202fde733dd7ca826ba372fba9fc034ac53ac264f7"} Jul 24 22:20:54.492415 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:54.492378 2554 generic.go:356] "Generic (PLEG): container finished" podID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerID="ee3623dcd17ac7b61d2603b0ef0db006527e15c554f21818a65b45f3c8c6ad11" exitCode=0 Jul 24 22:20:54.492805 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:54.492441 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" event={"ID":"6c56aa46-5936-400e-8c57-d2fb530392b1","Type":"ContainerDied","Data":"ee3623dcd17ac7b61d2603b0ef0db006527e15c554f21818a65b45f3c8c6ad11"} Jul 24 22:20:55.498801 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:55.498766 2554 generic.go:356] "Generic (PLEG): container finished" podID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerID="8af541bd29e6c4efcd85cfac42c24d4c8285cf68790ce65121356ac5aada6e33" exitCode=0 Jul 24 22:20:55.499238 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:55.498831 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" event={"ID":"6c56aa46-5936-400e-8c57-d2fb530392b1","Type":"ContainerDied","Data":"8af541bd29e6c4efcd85cfac42c24d4c8285cf68790ce65121356ac5aada6e33"} Jul 24 22:20:56.635043 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.635021 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:56.689650 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.689623 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpx26\" (UniqueName: \"kubernetes.io/projected/6c56aa46-5936-400e-8c57-d2fb530392b1-kube-api-access-fpx26\") pod \"6c56aa46-5936-400e-8c57-d2fb530392b1\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " Jul 24 22:20:56.689771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.689659 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-util\") pod \"6c56aa46-5936-400e-8c57-d2fb530392b1\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " Jul 24 22:20:56.689771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.689682 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-bundle\") pod \"6c56aa46-5936-400e-8c57-d2fb530392b1\" (UID: \"6c56aa46-5936-400e-8c57-d2fb530392b1\") " Jul 24 22:20:56.690397 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.690342 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-bundle" (OuterVolumeSpecName: "bundle") pod "6c56aa46-5936-400e-8c57-d2fb530392b1" (UID: "6c56aa46-5936-400e-8c57-d2fb530392b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:20:56.691710 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.691684 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c56aa46-5936-400e-8c57-d2fb530392b1-kube-api-access-fpx26" (OuterVolumeSpecName: "kube-api-access-fpx26") pod "6c56aa46-5936-400e-8c57-d2fb530392b1" (UID: "6c56aa46-5936-400e-8c57-d2fb530392b1"). InnerVolumeSpecName "kube-api-access-fpx26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:20:56.694975 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.694953 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-util" (OuterVolumeSpecName: "util") pod "6c56aa46-5936-400e-8c57-d2fb530392b1" (UID: "6c56aa46-5936-400e-8c57-d2fb530392b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:20:56.791235 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.791137 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpx26\" (UniqueName: \"kubernetes.io/projected/6c56aa46-5936-400e-8c57-d2fb530392b1-kube-api-access-fpx26\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:56.791235 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.791186 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:56.791235 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:56.791197 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c56aa46-5936-400e-8c57-d2fb530392b1-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:20:57.509389 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:57.509301 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" event={"ID":"6c56aa46-5936-400e-8c57-d2fb530392b1","Type":"ContainerDied","Data":"788ab9a67a85d88fb497a7202fde733dd7ca826ba372fba9fc034ac53ac264f7"} Jul 24 22:20:57.509389 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:57.509348 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft554k" Jul 24 22:20:57.509589 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:20:57.509351 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="788ab9a67a85d88fb497a7202fde733dd7ca826ba372fba9fc034ac53ac264f7" Jul 24 22:21:06.679729 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679692 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb"] Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679936 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="pull" Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679956 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="pull" Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679968 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="util" Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679973 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="util" Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679980 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="extract" Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.679985 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="extract" Jul 24 22:21:06.680226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.680037 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c56aa46-5936-400e-8c57-d2fb530392b1" containerName="extract" Jul 24 22:21:06.682372 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.682355 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.687165 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.687116 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 22:21:06.687319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.687187 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 22:21:06.687319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.687225 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 22:21:06.687319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.687286 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-v4rcb\"" Jul 24 22:21:06.687470 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.687384 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 22:21:06.709206 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.709181 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb"] Jul 24 22:21:06.761084 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.761059 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-manifests\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.761223 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.761088 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hlm\" (UniqueName: \"kubernetes.io/projected/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-kube-api-access-82hlm\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.761223 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.761123 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.761223 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.761208 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-tmp\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.761360 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.761226 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.862010 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.861983 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.862110 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.862023 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-tmp\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.862110 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.862048 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.862251 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.862204 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-manifests\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.862309 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.862256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82hlm\" (UniqueName: \"kubernetes.io/projected/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-kube-api-access-82hlm\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.862537 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.862509 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-manifests\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.864548 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.864522 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-tmp\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.864657 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.864641 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.864743 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.864723 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.875814 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.875795 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hlm\" (UniqueName: \"kubernetes.io/projected/48e11fca-1e70-41de-b8c3-5ebe02ea63a5-kube-api-access-82hlm\") pod \"opendatahub-operator-controller-manager-854fb8676-ptctb\" (UID: \"48e11fca-1e70-41de-b8c3-5ebe02ea63a5\") " pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:06.992783 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:06.992734 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:07.125130 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:07.125097 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb"] Jul 24 22:21:07.128756 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:21:07.128726 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e11fca_1e70_41de_b8c3_5ebe02ea63a5.slice/crio-a164f6a76a3fb03b37d9aeeac6ba20d94526cfb42bc823203afa79f0249ecb86 WatchSource:0}: Error finding container a164f6a76a3fb03b37d9aeeac6ba20d94526cfb42bc823203afa79f0249ecb86: Status 404 returned error can't find the container with id a164f6a76a3fb03b37d9aeeac6ba20d94526cfb42bc823203afa79f0249ecb86 Jul 24 22:21:07.549545 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:07.549511 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" event={"ID":"48e11fca-1e70-41de-b8c3-5ebe02ea63a5","Type":"ContainerStarted","Data":"a164f6a76a3fb03b37d9aeeac6ba20d94526cfb42bc823203afa79f0249ecb86"} Jul 24 22:21:10.567255 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:10.567216 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" event={"ID":"48e11fca-1e70-41de-b8c3-5ebe02ea63a5","Type":"ContainerStarted","Data":"1675e2eaa8d917b6181f8ddf20a9f05fee35be63d067f178efc7ca93dbc94a69"} Jul 24 22:21:11.573199 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:11.573104 2554 generic.go:356] "Generic (PLEG): container finished" podID="48e11fca-1e70-41de-b8c3-5ebe02ea63a5" containerID="1675e2eaa8d917b6181f8ddf20a9f05fee35be63d067f178efc7ca93dbc94a69" exitCode=0 Jul 24 22:21:11.573199 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:11.573176 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" event={"ID":"48e11fca-1e70-41de-b8c3-5ebe02ea63a5","Type":"ContainerDied","Data":"1675e2eaa8d917b6181f8ddf20a9f05fee35be63d067f178efc7ca93dbc94a69"} Jul 24 22:21:12.579926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:12.579893 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" event={"ID":"48e11fca-1e70-41de-b8c3-5ebe02ea63a5","Type":"ContainerStarted","Data":"0ca8ea393311f77f709b9b020b2cb4042667222127deead93bc37833bab25366"} Jul 24 22:21:12.580409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:12.580065 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:12.606300 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:12.606251 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" podStartSLOduration=2.177400554 podStartE2EDuration="6.606233997s" totalImagesPullingTime="3.361972003s" totalInitContainerRuntime="1.06686144s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:21:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:21:12.604121829 +0000 UTC m=+369.990939876" watchObservedRunningTime="2026-07-24 22:21:12.606233997 +0000 UTC m=+369.993052043" Jul 24 22:21:23.586531 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:23.586501 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-854fb8676-ptctb" Jul 24 22:21:25.821782 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.821753 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r"] Jul 24 22:21:25.824932 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.824915 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:25.828391 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.828351 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jcnrj\"" Jul 24 22:21:25.829725 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.829697 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 22:21:25.830007 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.829987 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 22:21:25.830133 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.830110 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 22:21:25.844249 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.844229 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r"] Jul 24 22:21:25.901906 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.901878 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/614a5ea8-1e97-40ac-b72a-b7748929af94-manager-config\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:25.901999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.901914 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc97q\" (UniqueName: \"kubernetes.io/projected/614a5ea8-1e97-40ac-b72a-b7748929af94-kube-api-access-fc97q\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:25.901999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.901932 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/614a5ea8-1e97-40ac-b72a-b7748929af94-metrics-cert\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:25.901999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:25.901989 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/614a5ea8-1e97-40ac-b72a-b7748929af94-cert\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.003378 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.003349 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/614a5ea8-1e97-40ac-b72a-b7748929af94-manager-config\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.003510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.003383 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc97q\" (UniqueName: \"kubernetes.io/projected/614a5ea8-1e97-40ac-b72a-b7748929af94-kube-api-access-fc97q\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.003510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.003402 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/614a5ea8-1e97-40ac-b72a-b7748929af94-metrics-cert\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.003510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.003424 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/614a5ea8-1e97-40ac-b72a-b7748929af94-cert\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.004003 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.003979 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/614a5ea8-1e97-40ac-b72a-b7748929af94-manager-config\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.006361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.006339 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/614a5ea8-1e97-40ac-b72a-b7748929af94-cert\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.006432 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.006339 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/614a5ea8-1e97-40ac-b72a-b7748929af94-metrics-cert\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.011973 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.011942 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc97q\" (UniqueName: \"kubernetes.io/projected/614a5ea8-1e97-40ac-b72a-b7748929af94-kube-api-access-fc97q\") pod \"lws-controller-manager-7b958b6f7b-qgb8r\" (UID: \"614a5ea8-1e97-40ac-b72a-b7748929af94\") " pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.136924 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.136864 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:26.271225 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.271143 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r"] Jul 24 22:21:26.275169 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:21:26.275112 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614a5ea8_1e97_40ac_b72a_b7748929af94.slice/crio-cb68ae043fc282edeaa5c6868d10eb28bdd084fe1ff2ec050818ab453bda296e WatchSource:0}: Error finding container cb68ae043fc282edeaa5c6868d10eb28bdd084fe1ff2ec050818ab453bda296e: Status 404 returned error can't find the container with id cb68ae043fc282edeaa5c6868d10eb28bdd084fe1ff2ec050818ab453bda296e Jul 24 22:21:26.645734 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:26.645699 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" event={"ID":"614a5ea8-1e97-40ac-b72a-b7748929af94","Type":"ContainerStarted","Data":"cb68ae043fc282edeaa5c6868d10eb28bdd084fe1ff2ec050818ab453bda296e"} Jul 24 22:21:28.657324 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:28.657288 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" event={"ID":"614a5ea8-1e97-40ac-b72a-b7748929af94","Type":"ContainerStarted","Data":"52e756856ca7020271fab2033c351667ebbffd65ae691d57162715bcc2392e10"} Jul 24 22:21:28.657705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:28.657404 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:21:28.673519 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:28.673465 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" podStartSLOduration=2.082863732 podStartE2EDuration="3.673449052s" totalImagesPullingTime="1.59058532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:21:28.67307796 +0000 UTC m=+386.059896006" watchObservedRunningTime="2026-07-24 22:21:28.673449052 +0000 UTC m=+386.060267100" Jul 24 22:21:33.045051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.045020 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq"] Jul 24 22:21:33.048480 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.048458 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.051332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.051308 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vpvjs\"" Jul 24 22:21:33.051485 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.051349 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 22:21:33.064422 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.064402 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq"] Jul 24 22:21:33.163037 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163014 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163171 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163044 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxfkp\" (UniqueName: \"kubernetes.io/projected/9549c3dc-c2ef-4c2e-a716-11d3338e3234-kube-api-access-pxfkp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163171 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163066 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163176 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163248 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163347 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163277 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163347 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163338 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.163405 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.163355 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264002 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.263955 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264011 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264038 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264064 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264086 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264106 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxfkp\" (UniqueName: \"kubernetes.io/projected/9549c3dc-c2ef-4c2e-a716-11d3338e3234-kube-api-access-pxfkp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264175 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264300 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264493 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264468 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264557 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264651 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264612 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264763 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264744 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.264823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.264755 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.265104 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.265083 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.266828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.266812 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.267058 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.267038 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.273859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.273815 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxfkp\" (UniqueName: \"kubernetes.io/projected/9549c3dc-c2ef-4c2e-a716-11d3338e3234-kube-api-access-pxfkp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.274074 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.274055 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9549c3dc-c2ef-4c2e-a716-11d3338e3234-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq\" (UID: \"9549c3dc-c2ef-4c2e-a716-11d3338e3234\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.358757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.358708 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:33.401574 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.401547 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct"] Jul 24 22:21:33.404999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.404965 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.407813 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.407791 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 22:21:33.408299 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.408282 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xmv6l\"" Jul 24 22:21:33.408584 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.408568 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 22:21:33.409187 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.408905 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 22:21:33.420696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.420654 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct"] Jul 24 22:21:33.508937 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.508775 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq"] Jul 24 22:21:33.511456 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:21:33.511434 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9549c3dc_c2ef_4c2e_a716_11d3338e3234.slice/crio-df0af30f33c5ea179115d3f39086c95d5e545ff12520c510ddc51f843fc63dcf WatchSource:0}: Error finding container df0af30f33c5ea179115d3f39086c95d5e545ff12520c510ddc51f843fc63dcf: Status 404 returned error can't find the container with id df0af30f33c5ea179115d3f39086c95d5e545ff12520c510ddc51f843fc63dcf Jul 24 22:21:33.568114 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568091 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.568232 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568118 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f55c2\" (UniqueName: \"kubernetes.io/projected/68dbf04d-ba00-4c10-bfd8-2e57db942711-kube-api-access-f55c2\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.568232 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568169 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.568345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568253 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dbf04d-ba00-4c10-bfd8-2e57db942711-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.568345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568298 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.568345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568323 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.568472 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.568365 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.668883 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.668861 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.668977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.668890 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dbf04d-ba00-4c10-bfd8-2e57db942711-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.668977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.668909 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.669057 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.669037 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.669104 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.669087 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.669214 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.669196 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.669294 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.669238 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f55c2\" (UniqueName: \"kubernetes.io/projected/68dbf04d-ba00-4c10-bfd8-2e57db942711-kube-api-access-f55c2\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.669908 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.669812 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.671522 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.671501 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.671686 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.671667 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.671809 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.671790 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/68dbf04d-ba00-4c10-bfd8-2e57db942711-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.671909 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.671888 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.679880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.679855 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" event={"ID":"9549c3dc-c2ef-4c2e-a716-11d3338e3234","Type":"ContainerStarted","Data":"df0af30f33c5ea179115d3f39086c95d5e545ff12520c510ddc51f843fc63dcf"} Jul 24 22:21:33.681045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.681014 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f55c2\" (UniqueName: \"kubernetes.io/projected/68dbf04d-ba00-4c10-bfd8-2e57db942711-kube-api-access-f55c2\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.681112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.681090 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68dbf04d-ba00-4c10-bfd8-2e57db942711-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-phlct\" (UID: \"68dbf04d-ba00-4c10-bfd8-2e57db942711\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.715730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.715710 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:33.846280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:33.846244 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct"] Jul 24 22:21:33.850548 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:21:33.850429 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dbf04d_ba00_4c10_bfd8_2e57db942711.slice/crio-abff00b3ababcc101948bf452b10a396ffb4ec1317210b4dd92b2b3ce73c8811 WatchSource:0}: Error finding container abff00b3ababcc101948bf452b10a396ffb4ec1317210b4dd92b2b3ce73c8811: Status 404 returned error can't find the container with id abff00b3ababcc101948bf452b10a396ffb4ec1317210b4dd92b2b3ce73c8811 Jul 24 22:21:34.685209 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:34.685168 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" event={"ID":"68dbf04d-ba00-4c10-bfd8-2e57db942711","Type":"ContainerStarted","Data":"abff00b3ababcc101948bf452b10a396ffb4ec1317210b4dd92b2b3ce73c8811"} Jul 24 22:21:37.504834 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.504774 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:21:37.505110 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.504882 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:21:37.508897 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.508866 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:21:37.509006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.508936 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:21:37.509006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.508982 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:21:37.703038 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.702999 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" event={"ID":"9549c3dc-c2ef-4c2e-a716-11d3338e3234","Type":"ContainerStarted","Data":"486fb3604dc8fdd7678660a615bef52b6caf44986e97c75857baf60e34e94a49"} Jul 24 22:21:37.704682 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.704658 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" event={"ID":"68dbf04d-ba00-4c10-bfd8-2e57db942711","Type":"ContainerStarted","Data":"3f2c60e0187c45a308b9c78094c03273ab454319c8f2386acc4213fc7f9d50e8"} Jul 24 22:21:37.704814 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.704800 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:37.728200 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.728138 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" podStartSLOduration=0.732931205 podStartE2EDuration="4.728124503s" totalImagesPullingTime="3.995193298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:21:37.726413352 +0000 UTC m=+395.113231398" watchObservedRunningTime="2026-07-24 22:21:37.728124503 +0000 UTC m=+395.114942549" Jul 24 22:21:37.747883 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:37.747829 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" podStartSLOduration=1.095237644 podStartE2EDuration="4.747810456s" totalImagesPullingTime="3.652572812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:21:37.746337772 +0000 UTC m=+395.133155819" watchObservedRunningTime="2026-07-24 22:21:37.747810456 +0000 UTC m=+395.134628493" Jul 24 22:21:38.359690 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:38.359644 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:38.365238 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:38.365206 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:38.709984 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:38.709949 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:38.711034 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:38.711008 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq" Jul 24 22:21:38.711702 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:38.711683 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-phlct" Jul 24 22:21:39.664482 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:21:39.664441 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b958b6f7b-qgb8r" Jul 24 22:22:42.465852 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.465826 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d"] Jul 24 22:22:42.467889 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.467875 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.470429 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.470409 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 22:22:42.470547 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.470435 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 22:22:42.470997 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.470962 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bckgb\"" Jul 24 22:22:42.478291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.478268 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d"] Jul 24 22:22:42.562693 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.562673 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.562786 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.562707 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.562786 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.562753 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r7fg\" (UniqueName: \"kubernetes.io/projected/b1309221-e4a0-403c-8fcd-a152c3fa9c75-kube-api-access-7r7fg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.663981 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.663949 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.664069 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.663989 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.664069 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.664022 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r7fg\" (UniqueName: \"kubernetes.io/projected/b1309221-e4a0-403c-8fcd-a152c3fa9c75-kube-api-access-7r7fg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.664388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.664368 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.664429 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.664383 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.672748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.672731 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r7fg\" (UniqueName: \"kubernetes.io/projected/b1309221-e4a0-403c-8fcd-a152c3fa9c75-kube-api-access-7r7fg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.778985 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.778940 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:42.901771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.901750 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d"] Jul 24 22:22:42.903762 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:22:42.903736 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1309221_e4a0_403c_8fcd_a152c3fa9c75.slice/crio-a89327e85ae3057325b337521e1affd6cc13030858a81464a5770b38c537f09f WatchSource:0}: Error finding container a89327e85ae3057325b337521e1affd6cc13030858a81464a5770b38c537f09f: Status 404 returned error can't find the container with id a89327e85ae3057325b337521e1affd6cc13030858a81464a5770b38c537f09f Jul 24 22:22:42.974079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.974044 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" event={"ID":"b1309221-e4a0-403c-8fcd-a152c3fa9c75","Type":"ContainerStarted","Data":"abdd98c5a6316774b67bd9afe15c1d9de1f9d4b2549d03a76f98684600ce40d7"} Jul 24 22:22:42.974214 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:42.974100 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" event={"ID":"b1309221-e4a0-403c-8fcd-a152c3fa9c75","Type":"ContainerStarted","Data":"a89327e85ae3057325b337521e1affd6cc13030858a81464a5770b38c537f09f"} Jul 24 22:22:43.065839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.065780 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2"] Jul 24 22:22:43.068007 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.067989 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.077307 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.077289 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2"] Jul 24 22:22:43.166798 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.166772 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdd9f\" (UniqueName: \"kubernetes.io/projected/2b0065ae-1aeb-410b-b990-520ac3ac969a-kube-api-access-bdd9f\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.166899 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.166810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.166899 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.166828 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.267940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.267920 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdd9f\" (UniqueName: \"kubernetes.io/projected/2b0065ae-1aeb-410b-b990-520ac3ac969a-kube-api-access-bdd9f\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.268045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.267953 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.268045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.267978 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.268385 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.268364 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.268431 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.268394 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.276968 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.276946 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdd9f\" (UniqueName: \"kubernetes.io/projected/2b0065ae-1aeb-410b-b990-520ac3ac969a-kube-api-access-bdd9f\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.377496 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.377437 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:43.497875 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.497853 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2"] Jul 24 22:22:43.499798 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:22:43.499773 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b0065ae_1aeb_410b_b990_520ac3ac969a.slice/crio-858af96350031e49c5ebbefa55259ec7fed8d6699cbaf5367fac4a34ed75aa52 WatchSource:0}: Error finding container 858af96350031e49c5ebbefa55259ec7fed8d6699cbaf5367fac4a34ed75aa52: Status 404 returned error can't find the container with id 858af96350031e49c5ebbefa55259ec7fed8d6699cbaf5367fac4a34ed75aa52 Jul 24 22:22:43.665071 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.665042 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7"] Jul 24 22:22:43.667225 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.667210 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.676031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.676009 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7"] Jul 24 22:22:43.770717 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.770690 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkrxk\" (UniqueName: \"kubernetes.io/projected/121883c5-a670-4cc0-a9bb-15260b6511a8-kube-api-access-hkrxk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.770807 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.770732 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.770807 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.770801 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.871982 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.871945 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkrxk\" (UniqueName: \"kubernetes.io/projected/121883c5-a670-4cc0-a9bb-15260b6511a8-kube-api-access-hkrxk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.872063 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.871993 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.872063 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.872028 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.872349 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.872333 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.872419 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.872374 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.880445 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.880428 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkrxk\" (UniqueName: \"kubernetes.io/projected/121883c5-a670-4cc0-a9bb-15260b6511a8-kube-api-access-hkrxk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:43.979574 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.979526 2554 generic.go:356] "Generic (PLEG): container finished" podID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerID="72ddd0aff7c145a0e11b9c3f01e7d15f51568b1698ded6196977489b5975afa2" exitCode=0 Jul 24 22:22:43.979658 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.979614 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" event={"ID":"2b0065ae-1aeb-410b-b990-520ac3ac969a","Type":"ContainerDied","Data":"72ddd0aff7c145a0e11b9c3f01e7d15f51568b1698ded6196977489b5975afa2"} Jul 24 22:22:43.979706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.979662 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" event={"ID":"2b0065ae-1aeb-410b-b990-520ac3ac969a","Type":"ContainerStarted","Data":"858af96350031e49c5ebbefa55259ec7fed8d6699cbaf5367fac4a34ed75aa52"} Jul 24 22:22:43.981170 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.981133 2554 generic.go:356] "Generic (PLEG): container finished" podID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerID="abdd98c5a6316774b67bd9afe15c1d9de1f9d4b2549d03a76f98684600ce40d7" exitCode=0 Jul 24 22:22:43.981254 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.981215 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" event={"ID":"b1309221-e4a0-403c-8fcd-a152c3fa9c75","Type":"ContainerDied","Data":"abdd98c5a6316774b67bd9afe15c1d9de1f9d4b2549d03a76f98684600ce40d7"} Jul 24 22:22:43.984412 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:43.984396 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:44.063749 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.063723 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5"] Jul 24 22:22:44.066342 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.066322 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.077313 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.077293 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5"] Jul 24 22:22:44.116956 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.116926 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7"] Jul 24 22:22:44.119407 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:22:44.119382 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121883c5_a670_4cc0_a9bb_15260b6511a8.slice/crio-4f719a42776f90e9c59747b44e88cb7deb7ffa4ebdf35b50a3a9e454b29b00bb WatchSource:0}: Error finding container 4f719a42776f90e9c59747b44e88cb7deb7ffa4ebdf35b50a3a9e454b29b00bb: Status 404 returned error can't find the container with id 4f719a42776f90e9c59747b44e88cb7deb7ffa4ebdf35b50a3a9e454b29b00bb Jul 24 22:22:44.179206 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.174728 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.179206 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.174788 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4c8f\" (UniqueName: \"kubernetes.io/projected/de5ffce6-2dbe-413e-bd33-ae9260474008-kube-api-access-b4c8f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.179206 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.174861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.275396 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.275340 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.275396 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.275375 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4c8f\" (UniqueName: \"kubernetes.io/projected/de5ffce6-2dbe-413e-bd33-ae9260474008-kube-api-access-b4c8f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.275515 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.275432 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.275750 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.275731 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.275750 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.275745 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.284175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.284128 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4c8f\" (UniqueName: \"kubernetes.io/projected/de5ffce6-2dbe-413e-bd33-ae9260474008-kube-api-access-b4c8f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.378622 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.378598 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:44.504616 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.504589 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5"] Jul 24 22:22:44.505823 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:22:44.505799 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5ffce6_2dbe_413e_bd33_ae9260474008.slice/crio-9e65d0bdceb423ab3648b7fb26943f844ce30c1a25a2155e3a3c6b7336b0a81a WatchSource:0}: Error finding container 9e65d0bdceb423ab3648b7fb26943f844ce30c1a25a2155e3a3c6b7336b0a81a: Status 404 returned error can't find the container with id 9e65d0bdceb423ab3648b7fb26943f844ce30c1a25a2155e3a3c6b7336b0a81a Jul 24 22:22:44.989193 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.989139 2554 generic.go:356] "Generic (PLEG): container finished" podID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerID="8b99b3a734bf91f767ace1868ab77ce3f031c2aa1a0d10bd20918984f67559a3" exitCode=0 Jul 24 22:22:44.989287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.989191 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" event={"ID":"de5ffce6-2dbe-413e-bd33-ae9260474008","Type":"ContainerDied","Data":"8b99b3a734bf91f767ace1868ab77ce3f031c2aa1a0d10bd20918984f67559a3"} Jul 24 22:22:44.989287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.989241 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" event={"ID":"de5ffce6-2dbe-413e-bd33-ae9260474008","Type":"ContainerStarted","Data":"9e65d0bdceb423ab3648b7fb26943f844ce30c1a25a2155e3a3c6b7336b0a81a"} Jul 24 22:22:44.991088 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.991065 2554 generic.go:356] "Generic (PLEG): container finished" podID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerID="a0bfeb0125de67792674898c2581b180e6c056178bd96112acb8e460575261b6" exitCode=0 Jul 24 22:22:44.991213 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.991167 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" event={"ID":"2b0065ae-1aeb-410b-b990-520ac3ac969a","Type":"ContainerDied","Data":"a0bfeb0125de67792674898c2581b180e6c056178bd96112acb8e460575261b6"} Jul 24 22:22:44.993034 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.993014 2554 generic.go:356] "Generic (PLEG): container finished" podID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerID="a898f764061fde3618e067dc88a0f8dcca4bb84bb8bcb77cb0f3b9cfb691b99b" exitCode=0 Jul 24 22:22:44.993116 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.993076 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" event={"ID":"b1309221-e4a0-403c-8fcd-a152c3fa9c75","Type":"ContainerDied","Data":"a898f764061fde3618e067dc88a0f8dcca4bb84bb8bcb77cb0f3b9cfb691b99b"} Jul 24 22:22:44.994546 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.994526 2554 generic.go:356] "Generic (PLEG): container finished" podID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerID="1f4a62695d8aaec7dfadbc0f178b017729a39a11622951072611be4c9f420af6" exitCode=0 Jul 24 22:22:44.994641 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.994580 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" event={"ID":"121883c5-a670-4cc0-a9bb-15260b6511a8","Type":"ContainerDied","Data":"1f4a62695d8aaec7dfadbc0f178b017729a39a11622951072611be4c9f420af6"} Jul 24 22:22:44.994641 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:44.994619 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" event={"ID":"121883c5-a670-4cc0-a9bb-15260b6511a8","Type":"ContainerStarted","Data":"4f719a42776f90e9c59747b44e88cb7deb7ffa4ebdf35b50a3a9e454b29b00bb"} Jul 24 22:22:46.000797 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.000768 2554 generic.go:356] "Generic (PLEG): container finished" podID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerID="0c7bfab16ba626c1084f4b4379b9822e73d2b1a67926a854d1a0b5c3354d797d" exitCode=0 Jul 24 22:22:46.001105 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.000856 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" event={"ID":"2b0065ae-1aeb-410b-b990-520ac3ac969a","Type":"ContainerDied","Data":"0c7bfab16ba626c1084f4b4379b9822e73d2b1a67926a854d1a0b5c3354d797d"} Jul 24 22:22:46.002821 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.002792 2554 generic.go:356] "Generic (PLEG): container finished" podID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerID="b0acb33e5d3eb81aaf97b020c2769684157869aa37619fe964aedf1c97da6a43" exitCode=0 Jul 24 22:22:46.002903 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.002869 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" event={"ID":"b1309221-e4a0-403c-8fcd-a152c3fa9c75","Type":"ContainerDied","Data":"b0acb33e5d3eb81aaf97b020c2769684157869aa37619fe964aedf1c97da6a43"} Jul 24 22:22:46.004625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.004605 2554 generic.go:356] "Generic (PLEG): container finished" podID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerID="cdd38bc608568c68ff80d2f3ae72230abf5b64b7579759187f134442046de458" exitCode=0 Jul 24 22:22:46.004745 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.004695 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" event={"ID":"121883c5-a670-4cc0-a9bb-15260b6511a8","Type":"ContainerDied","Data":"cdd38bc608568c68ff80d2f3ae72230abf5b64b7579759187f134442046de458"} Jul 24 22:22:46.006452 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.006430 2554 generic.go:356] "Generic (PLEG): container finished" podID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerID="da7eca61f4864ffd60c5e406893832952cee32ef9960956c455918795027a01a" exitCode=0 Jul 24 22:22:46.006521 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:46.006474 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" event={"ID":"de5ffce6-2dbe-413e-bd33-ae9260474008","Type":"ContainerDied","Data":"da7eca61f4864ffd60c5e406893832952cee32ef9960956c455918795027a01a"} Jul 24 22:22:47.013051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.013015 2554 generic.go:356] "Generic (PLEG): container finished" podID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerID="e2b51857e606f9404c24bc6b52175c578b959b589932edae7eea48a350254d29" exitCode=0 Jul 24 22:22:47.013496 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.013101 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" event={"ID":"121883c5-a670-4cc0-a9bb-15260b6511a8","Type":"ContainerDied","Data":"e2b51857e606f9404c24bc6b52175c578b959b589932edae7eea48a350254d29"} Jul 24 22:22:47.015052 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.015030 2554 generic.go:356] "Generic (PLEG): container finished" podID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerID="6c96b139a3e77f0f4bf5cdce9c3ff8d05b6b0100e98b1c93617d71d14adde713" exitCode=0 Jul 24 22:22:47.015172 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.015110 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" event={"ID":"de5ffce6-2dbe-413e-bd33-ae9260474008","Type":"ContainerDied","Data":"6c96b139a3e77f0f4bf5cdce9c3ff8d05b6b0100e98b1c93617d71d14adde713"} Jul 24 22:22:47.157512 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.157489 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:47.159963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.159942 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:47.200435 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.200409 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdd9f\" (UniqueName: \"kubernetes.io/projected/2b0065ae-1aeb-410b-b990-520ac3ac969a-kube-api-access-bdd9f\") pod \"2b0065ae-1aeb-410b-b990-520ac3ac969a\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " Jul 24 22:22:47.200621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.200441 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-util\") pod \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " Jul 24 22:22:47.200621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.200473 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-bundle\") pod \"2b0065ae-1aeb-410b-b990-520ac3ac969a\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " Jul 24 22:22:47.200621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.200518 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-bundle\") pod \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " Jul 24 22:22:47.200621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.200553 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r7fg\" (UniqueName: \"kubernetes.io/projected/b1309221-e4a0-403c-8fcd-a152c3fa9c75-kube-api-access-7r7fg\") pod \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\" (UID: \"b1309221-e4a0-403c-8fcd-a152c3fa9c75\") " Jul 24 22:22:47.200621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.200599 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-util\") pod \"2b0065ae-1aeb-410b-b990-520ac3ac969a\" (UID: \"2b0065ae-1aeb-410b-b990-520ac3ac969a\") " Jul 24 22:22:47.201218 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.201106 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-bundle" (OuterVolumeSpecName: "bundle") pod "2b0065ae-1aeb-410b-b990-520ac3ac969a" (UID: "2b0065ae-1aeb-410b-b990-520ac3ac969a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:47.201218 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.201183 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-bundle" (OuterVolumeSpecName: "bundle") pod "b1309221-e4a0-403c-8fcd-a152c3fa9c75" (UID: "b1309221-e4a0-403c-8fcd-a152c3fa9c75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:47.203389 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.203366 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b0065ae-1aeb-410b-b990-520ac3ac969a-kube-api-access-bdd9f" (OuterVolumeSpecName: "kube-api-access-bdd9f") pod "2b0065ae-1aeb-410b-b990-520ac3ac969a" (UID: "2b0065ae-1aeb-410b-b990-520ac3ac969a"). InnerVolumeSpecName "kube-api-access-bdd9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:22:47.203476 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.203400 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1309221-e4a0-403c-8fcd-a152c3fa9c75-kube-api-access-7r7fg" (OuterVolumeSpecName: "kube-api-access-7r7fg") pod "b1309221-e4a0-403c-8fcd-a152c3fa9c75" (UID: "b1309221-e4a0-403c-8fcd-a152c3fa9c75"). InnerVolumeSpecName "kube-api-access-7r7fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:22:47.209255 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.209231 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-util" (OuterVolumeSpecName: "util") pod "2b0065ae-1aeb-410b-b990-520ac3ac969a" (UID: "2b0065ae-1aeb-410b-b990-520ac3ac969a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:47.209626 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.209592 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-util" (OuterVolumeSpecName: "util") pod "b1309221-e4a0-403c-8fcd-a152c3fa9c75" (UID: "b1309221-e4a0-403c-8fcd-a152c3fa9c75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:47.302094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.302018 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdd9f\" (UniqueName: \"kubernetes.io/projected/2b0065ae-1aeb-410b-b990-520ac3ac969a-kube-api-access-bdd9f\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:47.302094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.302055 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:47.302094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.302067 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:47.302094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.302075 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1309221-e4a0-403c-8fcd-a152c3fa9c75-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:47.302094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.302082 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r7fg\" (UniqueName: \"kubernetes.io/projected/b1309221-e4a0-403c-8fcd-a152c3fa9c75-kube-api-access-7r7fg\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:47.302094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:47.302090 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2b0065ae-1aeb-410b-b990-520ac3ac969a-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:48.021328 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.021304 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" Jul 24 22:22:48.021328 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.021322 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2" event={"ID":"2b0065ae-1aeb-410b-b990-520ac3ac969a","Type":"ContainerDied","Data":"858af96350031e49c5ebbefa55259ec7fed8d6699cbaf5367fac4a34ed75aa52"} Jul 24 22:22:48.021696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.021385 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="858af96350031e49c5ebbefa55259ec7fed8d6699cbaf5367fac4a34ed75aa52" Jul 24 22:22:48.023268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.023240 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" Jul 24 22:22:48.023268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.023249 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d" event={"ID":"b1309221-e4a0-403c-8fcd-a152c3fa9c75","Type":"ContainerDied","Data":"a89327e85ae3057325b337521e1affd6cc13030858a81464a5770b38c537f09f"} Jul 24 22:22:48.023428 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.023292 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a89327e85ae3057325b337521e1affd6cc13030858a81464a5770b38c537f09f" Jul 24 22:22:48.160543 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.160523 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:22:48.187232 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.187212 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:48.207797 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.207773 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4c8f\" (UniqueName: \"kubernetes.io/projected/de5ffce6-2dbe-413e-bd33-ae9260474008-kube-api-access-b4c8f\") pod \"de5ffce6-2dbe-413e-bd33-ae9260474008\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " Jul 24 22:22:48.207940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.207850 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-util\") pod \"de5ffce6-2dbe-413e-bd33-ae9260474008\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " Jul 24 22:22:48.207940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.207892 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-bundle\") pod \"de5ffce6-2dbe-413e-bd33-ae9260474008\" (UID: \"de5ffce6-2dbe-413e-bd33-ae9260474008\") " Jul 24 22:22:48.208618 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.208589 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-bundle" (OuterVolumeSpecName: "bundle") pod "de5ffce6-2dbe-413e-bd33-ae9260474008" (UID: "de5ffce6-2dbe-413e-bd33-ae9260474008"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:48.210008 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.209985 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5ffce6-2dbe-413e-bd33-ae9260474008-kube-api-access-b4c8f" (OuterVolumeSpecName: "kube-api-access-b4c8f") pod "de5ffce6-2dbe-413e-bd33-ae9260474008" (UID: "de5ffce6-2dbe-413e-bd33-ae9260474008"). InnerVolumeSpecName "kube-api-access-b4c8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:22:48.213087 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.213061 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-util" (OuterVolumeSpecName: "util") pod "de5ffce6-2dbe-413e-bd33-ae9260474008" (UID: "de5ffce6-2dbe-413e-bd33-ae9260474008"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:48.308585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.308523 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkrxk\" (UniqueName: \"kubernetes.io/projected/121883c5-a670-4cc0-a9bb-15260b6511a8-kube-api-access-hkrxk\") pod \"121883c5-a670-4cc0-a9bb-15260b6511a8\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " Jul 24 22:22:48.308684 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.308591 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-util\") pod \"121883c5-a670-4cc0-a9bb-15260b6511a8\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " Jul 24 22:22:48.308684 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.308652 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-bundle\") pod \"121883c5-a670-4cc0-a9bb-15260b6511a8\" (UID: \"121883c5-a670-4cc0-a9bb-15260b6511a8\") " Jul 24 22:22:48.308839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.308822 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4c8f\" (UniqueName: \"kubernetes.io/projected/de5ffce6-2dbe-413e-bd33-ae9260474008-kube-api-access-b4c8f\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:48.308905 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.308845 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:48.308905 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.308858 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de5ffce6-2dbe-413e-bd33-ae9260474008-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:48.309173 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.309128 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-bundle" (OuterVolumeSpecName: "bundle") pod "121883c5-a670-4cc0-a9bb-15260b6511a8" (UID: "121883c5-a670-4cc0-a9bb-15260b6511a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:48.310575 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.310556 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121883c5-a670-4cc0-a9bb-15260b6511a8-kube-api-access-hkrxk" (OuterVolumeSpecName: "kube-api-access-hkrxk") pod "121883c5-a670-4cc0-a9bb-15260b6511a8" (UID: "121883c5-a670-4cc0-a9bb-15260b6511a8"). InnerVolumeSpecName "kube-api-access-hkrxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:22:48.313889 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.313854 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-util" (OuterVolumeSpecName: "util") pod "121883c5-a670-4cc0-a9bb-15260b6511a8" (UID: "121883c5-a670-4cc0-a9bb-15260b6511a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:22:48.409291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.409262 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkrxk\" (UniqueName: \"kubernetes.io/projected/121883c5-a670-4cc0-a9bb-15260b6511a8-kube-api-access-hkrxk\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:48.409291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.409285 2554 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-util\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:48.409291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:48.409295 2554 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/121883c5-a670-4cc0-a9bb-15260b6511a8-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:22:49.029493 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:49.029411 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" Jul 24 22:22:49.029493 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:49.029431 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7" event={"ID":"121883c5-a670-4cc0-a9bb-15260b6511a8","Type":"ContainerDied","Data":"4f719a42776f90e9c59747b44e88cb7deb7ffa4ebdf35b50a3a9e454b29b00bb"} Jul 24 22:22:49.029493 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:49.029469 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f719a42776f90e9c59747b44e88cb7deb7ffa4ebdf35b50a3a9e454b29b00bb" Jul 24 22:22:49.031257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:49.031226 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" event={"ID":"de5ffce6-2dbe-413e-bd33-ae9260474008","Type":"ContainerDied","Data":"9e65d0bdceb423ab3648b7fb26943f844ce30c1a25a2155e3a3c6b7336b0a81a"} Jul 24 22:22:49.031414 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:49.031267 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e65d0bdceb423ab3648b7fb26943f844ce30c1a25a2155e3a3c6b7336b0a81a" Jul 24 22:22:49.031414 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:22:49.031232 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5" Jul 24 22:23:14.425070 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:14.425039 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78dd7d4468-klt8g"] Jul 24 22:23:39.444616 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.444549 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78dd7d4468-klt8g" podUID="97b4da9f-9712-4abb-b16e-51c52129f449" containerName="console" containerID="cri-o://23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9" gracePeriod=15 Jul 24 22:23:39.699743 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.699688 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:23:39.816553 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.816514 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4dxh\" (UniqueName: \"kubernetes.io/projected/97b4da9f-9712-4abb-b16e-51c52129f449-kube-api-access-q4dxh\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.816915 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.816897 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-service-ca\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.817014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.816928 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-console-config\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.817014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.816983 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-oauth-config\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.817014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817004 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-oauth-serving-cert\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.817141 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817029 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-trusted-ca-bundle\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.817141 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817064 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-serving-cert\") pod \"97b4da9f-9712-4abb-b16e-51c52129f449\" (UID: \"97b4da9f-9712-4abb-b16e-51c52129f449\") " Jul 24 22:23:39.817359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817331 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-service-ca" (OuterVolumeSpecName: "service-ca") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:23:39.817396 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817353 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:23:39.817743 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817705 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:23:39.817743 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.817726 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-console-config" (OuterVolumeSpecName: "console-config") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:23:39.818872 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.818839 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b4da9f-9712-4abb-b16e-51c52129f449-kube-api-access-q4dxh" (OuterVolumeSpecName: "kube-api-access-q4dxh") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "kube-api-access-q4dxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:23:39.819342 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.819323 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:23:39.819542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.819524 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "97b4da9f-9712-4abb-b16e-51c52129f449" (UID: "97b4da9f-9712-4abb-b16e-51c52129f449"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:23:39.917821 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917780 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-oauth-config\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:39.917821 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917809 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-oauth-serving-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:39.917821 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917818 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-trusted-ca-bundle\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:39.917821 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917826 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97b4da9f-9712-4abb-b16e-51c52129f449-console-serving-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:39.918085 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917836 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4dxh\" (UniqueName: \"kubernetes.io/projected/97b4da9f-9712-4abb-b16e-51c52129f449-kube-api-access-q4dxh\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:39.918085 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917844 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-service-ca\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:39.918085 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:39.917852 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97b4da9f-9712-4abb-b16e-51c52129f449-console-config\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:23:40.257504 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.257473 2554 generic.go:356] "Generic (PLEG): container finished" podID="97b4da9f-9712-4abb-b16e-51c52129f449" containerID="23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9" exitCode=0 Jul 24 22:23:40.257667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.257515 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dd7d4468-klt8g" event={"ID":"97b4da9f-9712-4abb-b16e-51c52129f449","Type":"ContainerDied","Data":"23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9"} Jul 24 22:23:40.257667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.257555 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78dd7d4468-klt8g" event={"ID":"97b4da9f-9712-4abb-b16e-51c52129f449","Type":"ContainerDied","Data":"1a925cc851a88d2dc9a81bd293c63613bb13e716d36cfab371e625d6d9ace510"} Jul 24 22:23:40.257667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.257574 2554 scope.go:117] "RemoveContainer" containerID="23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9" Jul 24 22:23:40.257667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.257518 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78dd7d4468-klt8g" Jul 24 22:23:40.266378 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.266362 2554 scope.go:117] "RemoveContainer" containerID="23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9" Jul 24 22:23:40.266615 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:23:40.266598 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9\": container with ID starting with 23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9 not found: ID does not exist" containerID="23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9" Jul 24 22:23:40.266662 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.266622 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9"} err="failed to get container status \"23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9\": rpc error: code = NotFound desc = could not find container \"23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9\": container with ID starting with 23f7a6c181dc3899a324742d363b0331516eaf09162354c0ca57db9ac8deaff9 not found: ID does not exist" Jul 24 22:23:40.278933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.278907 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78dd7d4468-klt8g"] Jul 24 22:23:40.283205 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:40.283179 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78dd7d4468-klt8g"] Jul 24 22:23:41.180656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:23:41.180618 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97b4da9f-9712-4abb-b16e-51c52129f449" path="/var/lib/kubelet/pods/97b4da9f-9712-4abb-b16e-51c52129f449/volumes" Jul 24 22:25:21.415612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415576 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cbc94"] Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415821 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415831 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415840 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415846 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415855 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415860 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415868 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415873 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415879 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415884 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415891 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415896 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415902 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415907 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415913 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415919 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415927 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415931 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="util" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415936 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415940 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415946 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415951 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415960 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97b4da9f-9712-4abb-b16e-51c52129f449" containerName="console" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415973 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="97b4da9f-9712-4abb-b16e-51c52129f449" containerName="console" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415979 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.415983 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="pull" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.416027 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1309221-e4a0-403c-8fcd-a152c3fa9c75" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.416037 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="97b4da9f-9712-4abb-b16e-51c52129f449" containerName="console" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.416043 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="de5ffce6-2dbe-413e-bd33-ae9260474008" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.416048 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b0065ae-1aeb-410b-b990-520ac3ac969a" containerName="extract" Jul 24 22:25:21.417967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.416053 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="121883c5-a670-4cc0-a9bb-15260b6511a8" containerName="extract" Jul 24 22:25:21.419062 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.419047 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.421736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.421703 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8l4rx\"" Jul 24 22:25:21.421866 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.421790 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 22:25:21.429666 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.429643 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cbc94"] Jul 24 22:25:21.464500 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.464473 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a1f037-a1ee-4488-968c-ff384c53287d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cbc94\" (UID: \"19a1f037-a1ee-4488-968c-ff384c53287d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.464605 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.464519 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpmvg\" (UniqueName: \"kubernetes.io/projected/19a1f037-a1ee-4488-968c-ff384c53287d-kube-api-access-vpmvg\") pod \"payload-processing-5cc6cd444b-cbc94\" (UID: \"19a1f037-a1ee-4488-968c-ff384c53287d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.565443 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.565411 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a1f037-a1ee-4488-968c-ff384c53287d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cbc94\" (UID: \"19a1f037-a1ee-4488-968c-ff384c53287d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.565585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.565478 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpmvg\" (UniqueName: \"kubernetes.io/projected/19a1f037-a1ee-4488-968c-ff384c53287d-kube-api-access-vpmvg\") pod \"payload-processing-5cc6cd444b-cbc94\" (UID: \"19a1f037-a1ee-4488-968c-ff384c53287d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.566075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.566054 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a1f037-a1ee-4488-968c-ff384c53287d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-cbc94\" (UID: \"19a1f037-a1ee-4488-968c-ff384c53287d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.574773 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.574756 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpmvg\" (UniqueName: \"kubernetes.io/projected/19a1f037-a1ee-4488-968c-ff384c53287d-kube-api-access-vpmvg\") pod \"payload-processing-5cc6cd444b-cbc94\" (UID: \"19a1f037-a1ee-4488-968c-ff384c53287d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.729965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.729882 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:21.853611 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.853589 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-cbc94"] Jul 24 22:25:21.855316 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:25:21.855293 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a1f037_a1ee_4488_968c_ff384c53287d.slice/crio-4e807bfc155292c9ec6c446856dbad11a0c1c6540d48397ea532593fb37f708f WatchSource:0}: Error finding container 4e807bfc155292c9ec6c446856dbad11a0c1c6540d48397ea532593fb37f708f: Status 404 returned error can't find the container with id 4e807bfc155292c9ec6c446856dbad11a0c1c6540d48397ea532593fb37f708f Jul 24 22:25:21.857313 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:21.857299 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 22:25:22.732797 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:22.732758 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" event={"ID":"19a1f037-a1ee-4488-968c-ff384c53287d","Type":"ContainerStarted","Data":"4e807bfc155292c9ec6c446856dbad11a0c1c6540d48397ea532593fb37f708f"} Jul 24 22:25:23.741204 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:23.741172 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" event={"ID":"19a1f037-a1ee-4488-968c-ff384c53287d","Type":"ContainerStarted","Data":"77cfbc3fad392eeac91eda1ca28f51966df05f36e25bf2b53034ca4af47af3fd"} Jul 24 22:25:23.741560 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:23.741286 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:23.758602 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:23.758558 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" podStartSLOduration=1.283553809 podStartE2EDuration="2.758544641s" totalImagesPullingTime="1.474990832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:25:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:25:23.75651905 +0000 UTC m=+621.143337096" watchObservedRunningTime="2026-07-24 22:25:23.758544641 +0000 UTC m=+621.145362709" Jul 24 22:25:34.747862 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:34.747829 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-cbc94" Jul 24 22:25:36.924899 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.924860 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp"] Jul 24 22:25:36.928648 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.928622 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:36.931739 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.931718 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v4bdf\"" Jul 24 22:25:36.931857 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.931740 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 22:25:36.931857 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.931716 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 22:25:36.932906 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.932886 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 24 22:25:36.938123 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:36.938100 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp"] Jul 24 22:25:37.080547 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.080494 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.080547 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.080548 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.080780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.080577 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f823cd0-538d-42c2-8458-57c74ddd4cb4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.080780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.080607 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zm8\" (UniqueName: \"kubernetes.io/projected/8f823cd0-538d-42c2-8458-57c74ddd4cb4-kube-api-access-m8zm8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.080780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.080681 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.080780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.080711 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181322 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181251 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181322 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181291 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181322 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181309 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f823cd0-538d-42c2-8458-57c74ddd4cb4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181564 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181330 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zm8\" (UniqueName: \"kubernetes.io/projected/8f823cd0-538d-42c2-8458-57c74ddd4cb4-kube-api-access-m8zm8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181564 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181499 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181564 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181537 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181721 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181692 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181867 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181846 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.181910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.181856 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.183697 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.183678 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f823cd0-538d-42c2-8458-57c74ddd4cb4-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.184003 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.183986 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f823cd0-538d-42c2-8458-57c74ddd4cb4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.189573 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.189556 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zm8\" (UniqueName: \"kubernetes.io/projected/8f823cd0-538d-42c2-8458-57c74ddd4cb4-kube-api-access-m8zm8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp\" (UID: \"8f823cd0-538d-42c2-8458-57c74ddd4cb4\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.241568 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.241537 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:37.373271 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.373248 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp"] Jul 24 22:25:37.375350 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:25:37.375325 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f823cd0_538d_42c2_8458_57c74ddd4cb4.slice/crio-0de3156099a115fdbca3eeac1f0bc7c32e54f21ccf769731648bfa7e1671c401 WatchSource:0}: Error finding container 0de3156099a115fdbca3eeac1f0bc7c32e54f21ccf769731648bfa7e1671c401: Status 404 returned error can't find the container with id 0de3156099a115fdbca3eeac1f0bc7c32e54f21ccf769731648bfa7e1671c401 Jul 24 22:25:37.815244 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:37.815209 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" event={"ID":"8f823cd0-538d-42c2-8458-57c74ddd4cb4","Type":"ContainerStarted","Data":"0de3156099a115fdbca3eeac1f0bc7c32e54f21ccf769731648bfa7e1671c401"} Jul 24 22:25:38.821496 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:38.821408 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" event={"ID":"8f823cd0-538d-42c2-8458-57c74ddd4cb4","Type":"ContainerStarted","Data":"b64b1e00708ac214986a5fdfdc9b3e6b52724aef5d0b112c72a13ad79062a1d9"} Jul 24 22:25:38.837462 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:38.837418 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" podStartSLOduration=1.727097943 podStartE2EDuration="2.837403267s" totalImagesPullingTime="1.110305324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:25:38.836618386 +0000 UTC m=+636.223436435" watchObservedRunningTime="2026-07-24 22:25:38.837403267 +0000 UTC m=+636.224221314" Jul 24 22:25:47.242582 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:47.242539 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:47.243081 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:47.242617 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:47.255779 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:47.255749 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:25:47.875073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:25:47.875040 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp" Jul 24 22:29:48.037273 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.037240 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-656df4d8c4-m6n99"] Jul 24 22:29:48.041520 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.041504 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.045810 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.045749 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 22:29:48.045810 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.045802 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 22:29:48.046009 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.045880 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-64zqd\"" Jul 24 22:29:48.048120 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.048096 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-m6n99"] Jul 24 22:29:48.068847 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.068825 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkbd4\" (UniqueName: \"kubernetes.io/projected/3041e4bc-674a-4301-88f1-f10336c509f6-kube-api-access-kkbd4\") pod \"maas-controller-656df4d8c4-m6n99\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.068950 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.068919 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3041e4bc-674a-4301-88f1-f10336c509f6-webhook-cert\") pod \"maas-controller-656df4d8c4-m6n99\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.170044 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.170019 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3041e4bc-674a-4301-88f1-f10336c509f6-webhook-cert\") pod \"maas-controller-656df4d8c4-m6n99\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.170188 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.170059 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkbd4\" (UniqueName: \"kubernetes.io/projected/3041e4bc-674a-4301-88f1-f10336c509f6-kube-api-access-kkbd4\") pod \"maas-controller-656df4d8c4-m6n99\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.172614 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.172594 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3041e4bc-674a-4301-88f1-f10336c509f6-webhook-cert\") pod \"maas-controller-656df4d8c4-m6n99\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.178759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.178734 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkbd4\" (UniqueName: \"kubernetes.io/projected/3041e4bc-674a-4301-88f1-f10336c509f6-kube-api-access-kkbd4\") pod \"maas-controller-656df4d8c4-m6n99\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.374398 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.374340 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:48.500737 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.500715 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-m6n99"] Jul 24 22:29:48.502315 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:29:48.502290 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3041e4bc_674a_4301_88f1_f10336c509f6.slice/crio-4e456f7edc99fc588dfde3262b0842b9d6af9704469be6c79117907a6698eadb WatchSource:0}: Error finding container 4e456f7edc99fc588dfde3262b0842b9d6af9704469be6c79117907a6698eadb: Status 404 returned error can't find the container with id 4e456f7edc99fc588dfde3262b0842b9d6af9704469be6c79117907a6698eadb Jul 24 22:29:48.929894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:48.929868 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656df4d8c4-m6n99" event={"ID":"3041e4bc-674a-4301-88f1-f10336c509f6","Type":"ContainerStarted","Data":"4e456f7edc99fc588dfde3262b0842b9d6af9704469be6c79117907a6698eadb"} Jul 24 22:29:50.941679 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:50.941648 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656df4d8c4-m6n99" event={"ID":"3041e4bc-674a-4301-88f1-f10336c509f6","Type":"ContainerStarted","Data":"b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e"} Jul 24 22:29:50.942022 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:50.941718 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:29:50.961162 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:29:50.961086 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-656df4d8c4-m6n99" podStartSLOduration=0.733740946 podStartE2EDuration="2.961072514s" totalImagesPullingTime="2.227331568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:29:50.958956343 +0000 UTC m=+888.345774413" watchObservedRunningTime="2026-07-24 22:29:50.961072514 +0000 UTC m=+888.347890559" Jul 24 22:30:00.140848 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.140763 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr"] Jul 24 22:30:00.144031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.144010 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" Jul 24 22:30:00.146801 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.146781 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcbdx\"" Jul 24 22:30:00.150964 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.150938 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr"] Jul 24 22:30:00.155362 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.155339 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" Jul 24 22:30:00.281948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.281924 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr"] Jul 24 22:30:00.283658 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:30:00.283629 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa46e4e8_10d9_47df_926b_1baa8367bbac.slice/crio-c94fa6066674cbd3bbe7a7f15374855980444a9a8462e853800de8c1d8e3269c WatchSource:0}: Error finding container c94fa6066674cbd3bbe7a7f15374855980444a9a8462e853800de8c1d8e3269c: Status 404 returned error can't find the container with id c94fa6066674cbd3bbe7a7f15374855980444a9a8462e853800de8c1d8e3269c Jul 24 22:30:00.988024 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:00.987960 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" event={"ID":"fa46e4e8-10d9-47df-926b-1baa8367bbac","Type":"ContainerStarted","Data":"c94fa6066674cbd3bbe7a7f15374855980444a9a8462e853800de8c1d8e3269c"} Jul 24 22:30:01.948677 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:01.948646 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:30:04.005123 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:04.005078 2554 generic.go:356] "Generic (PLEG): container finished" podID="fa46e4e8-10d9-47df-926b-1baa8367bbac" containerID="c17790af8c03b96b85cfbd2ef9b3e152301380d998f76cc958aa2042d37a7f54" exitCode=0 Jul 24 22:30:04.005520 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:04.005185 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" event={"ID":"fa46e4e8-10d9-47df-926b-1baa8367bbac","Type":"ContainerDied","Data":"c17790af8c03b96b85cfbd2ef9b3e152301380d998f76cc958aa2042d37a7f54"} Jul 24 22:30:05.133139 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:05.133115 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" Jul 24 22:30:06.015023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.014994 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" Jul 24 22:30:06.015213 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.015030 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748870-rg7hr" event={"ID":"fa46e4e8-10d9-47df-926b-1baa8367bbac","Type":"ContainerDied","Data":"c94fa6066674cbd3bbe7a7f15374855980444a9a8462e853800de8c1d8e3269c"} Jul 24 22:30:06.015213 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.015072 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c94fa6066674cbd3bbe7a7f15374855980444a9a8462e853800de8c1d8e3269c" Jul 24 22:30:06.581582 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.581542 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-m6n99"] Jul 24 22:30:06.582169 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.581777 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-656df4d8c4-m6n99" podUID="3041e4bc-674a-4301-88f1-f10336c509f6" containerName="manager" containerID="cri-o://b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e" gracePeriod=30 Jul 24 22:30:06.840198 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.840108 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:30:06.917528 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.917496 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkbd4\" (UniqueName: \"kubernetes.io/projected/3041e4bc-674a-4301-88f1-f10336c509f6-kube-api-access-kkbd4\") pod \"3041e4bc-674a-4301-88f1-f10336c509f6\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " Jul 24 22:30:06.917696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.917619 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3041e4bc-674a-4301-88f1-f10336c509f6-webhook-cert\") pod \"3041e4bc-674a-4301-88f1-f10336c509f6\" (UID: \"3041e4bc-674a-4301-88f1-f10336c509f6\") " Jul 24 22:30:06.919733 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.919703 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3041e4bc-674a-4301-88f1-f10336c509f6-kube-api-access-kkbd4" (OuterVolumeSpecName: "kube-api-access-kkbd4") pod "3041e4bc-674a-4301-88f1-f10336c509f6" (UID: "3041e4bc-674a-4301-88f1-f10336c509f6"). InnerVolumeSpecName "kube-api-access-kkbd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:30:06.919733 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:06.919713 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3041e4bc-674a-4301-88f1-f10336c509f6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3041e4bc-674a-4301-88f1-f10336c509f6" (UID: "3041e4bc-674a-4301-88f1-f10336c509f6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:30:07.018175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.018128 2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3041e4bc-674a-4301-88f1-f10336c509f6-webhook-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:30:07.018175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.018173 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkbd4\" (UniqueName: \"kubernetes.io/projected/3041e4bc-674a-4301-88f1-f10336c509f6-kube-api-access-kkbd4\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:30:07.021076 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.021047 2554 generic.go:356] "Generic (PLEG): container finished" podID="3041e4bc-674a-4301-88f1-f10336c509f6" containerID="b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e" exitCode=0 Jul 24 22:30:07.021210 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.021106 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656df4d8c4-m6n99" Jul 24 22:30:07.021210 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.021132 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656df4d8c4-m6n99" event={"ID":"3041e4bc-674a-4301-88f1-f10336c509f6","Type":"ContainerDied","Data":"b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e"} Jul 24 22:30:07.021210 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.021207 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656df4d8c4-m6n99" event={"ID":"3041e4bc-674a-4301-88f1-f10336c509f6","Type":"ContainerDied","Data":"4e456f7edc99fc588dfde3262b0842b9d6af9704469be6c79117907a6698eadb"} Jul 24 22:30:07.021359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.021232 2554 scope.go:117] "RemoveContainer" containerID="b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e" Jul 24 22:30:07.029937 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.029921 2554 scope.go:117] "RemoveContainer" containerID="b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e" Jul 24 22:30:07.030260 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:30:07.030243 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e\": container with ID starting with b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e not found: ID does not exist" containerID="b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e" Jul 24 22:30:07.030323 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.030268 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e"} err="failed to get container status \"b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e\": rpc error: code = NotFound desc = could not find container \"b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e\": container with ID starting with b3f5ce80c3ba3104eb1902618042ac426facc7f63feac788e65d165d86bb555e not found: ID does not exist" Jul 24 22:30:07.042904 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.042879 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-m6n99"] Jul 24 22:30:07.046709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.046686 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-m6n99"] Jul 24 22:30:07.175120 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.175077 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3041e4bc-674a-4301-88f1-f10336c509f6" path="/var/lib/kubelet/pods/3041e4bc-674a-4301-88f1-f10336c509f6/volumes" Jul 24 22:30:07.807970 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.807923 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-656df4d8c4-g286b"] Jul 24 22:30:07.808461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.808277 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3041e4bc-674a-4301-88f1-f10336c509f6" containerName="manager" Jul 24 22:30:07.808461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.808291 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3041e4bc-674a-4301-88f1-f10336c509f6" containerName="manager" Jul 24 22:30:07.808461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.808300 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa46e4e8-10d9-47df-926b-1baa8367bbac" containerName="cleanup" Jul 24 22:30:07.808461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.808308 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa46e4e8-10d9-47df-926b-1baa8367bbac" containerName="cleanup" Jul 24 22:30:07.808461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.808377 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa46e4e8-10d9-47df-926b-1baa8367bbac" containerName="cleanup" Jul 24 22:30:07.808461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.808388 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3041e4bc-674a-4301-88f1-f10336c509f6" containerName="manager" Jul 24 22:30:07.812525 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.812506 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:07.816573 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.816544 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 22:30:07.816573 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.816570 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 22:30:07.816736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.816548 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-64zqd\"" Jul 24 22:30:07.822081 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.822057 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-g286b"] Jul 24 22:30:07.925118 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.925066 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d0e7fa-1e6b-407a-b972-ceac7be0480b-webhook-cert\") pod \"maas-controller-656df4d8c4-g286b\" (UID: \"e3d0e7fa-1e6b-407a-b972-ceac7be0480b\") " pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:07.925295 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:07.925168 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxpjx\" (UniqueName: \"kubernetes.io/projected/e3d0e7fa-1e6b-407a-b972-ceac7be0480b-kube-api-access-lxpjx\") pod \"maas-controller-656df4d8c4-g286b\" (UID: \"e3d0e7fa-1e6b-407a-b972-ceac7be0480b\") " pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:08.025868 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:08.025837 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxpjx\" (UniqueName: \"kubernetes.io/projected/e3d0e7fa-1e6b-407a-b972-ceac7be0480b-kube-api-access-lxpjx\") pod \"maas-controller-656df4d8c4-g286b\" (UID: \"e3d0e7fa-1e6b-407a-b972-ceac7be0480b\") " pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:08.026034 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:08.025904 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d0e7fa-1e6b-407a-b972-ceac7be0480b-webhook-cert\") pod \"maas-controller-656df4d8c4-g286b\" (UID: \"e3d0e7fa-1e6b-407a-b972-ceac7be0480b\") " pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:08.028592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:08.028568 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e3d0e7fa-1e6b-407a-b972-ceac7be0480b-webhook-cert\") pod \"maas-controller-656df4d8c4-g286b\" (UID: \"e3d0e7fa-1e6b-407a-b972-ceac7be0480b\") " pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:08.035321 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:08.035295 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxpjx\" (UniqueName: \"kubernetes.io/projected/e3d0e7fa-1e6b-407a-b972-ceac7be0480b-kube-api-access-lxpjx\") pod \"maas-controller-656df4d8c4-g286b\" (UID: \"e3d0e7fa-1e6b-407a-b972-ceac7be0480b\") " pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:08.123356 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:08.123285 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:08.251295 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:08.251272 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-656df4d8c4-g286b"] Jul 24 22:30:08.252785 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:30:08.252755 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d0e7fa_1e6b_407a_b972_ceac7be0480b.slice/crio-566843ea568c9ac25c57a8f474d2483cb5d8b072ff6906a963f69d9287c25948 WatchSource:0}: Error finding container 566843ea568c9ac25c57a8f474d2483cb5d8b072ff6906a963f69d9287c25948: Status 404 returned error can't find the container with id 566843ea568c9ac25c57a8f474d2483cb5d8b072ff6906a963f69d9287c25948 Jul 24 22:30:09.033318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:09.033283 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656df4d8c4-g286b" event={"ID":"e3d0e7fa-1e6b-407a-b972-ceac7be0480b","Type":"ContainerStarted","Data":"9565ea2de0ce63652837de49a13bd534a3c29732776f7554f8c3b72dc79f9540"} Jul 24 22:30:09.033752 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:09.033331 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-656df4d8c4-g286b" event={"ID":"e3d0e7fa-1e6b-407a-b972-ceac7be0480b","Type":"ContainerStarted","Data":"566843ea568c9ac25c57a8f474d2483cb5d8b072ff6906a963f69d9287c25948"} Jul 24 22:30:09.033752 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:09.033371 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:30:09.051540 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:09.051488 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-656df4d8c4-g286b" podStartSLOduration=1.6812315789999999 podStartE2EDuration="2.051472553s" totalImagesPullingTime="370.240974ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:30:09.050533779 +0000 UTC m=+906.437351847" watchObservedRunningTime="2026-07-24 22:30:09.051472553 +0000 UTC m=+906.438290601" Jul 24 22:30:20.040959 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:30:20.040922 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-656df4d8c4-g286b" Jul 24 22:38:45.604809 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.604773 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz"] Jul 24 22:38:45.608012 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.607989 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.610749 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.610717 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-94df6b07-kserve-self-signed-certs\"" Jul 24 22:38:45.618166 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.618129 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz"] Jul 24 22:38:45.745409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.745382 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-dshm\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.745516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.745417 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f286587-5131-41ed-a704-08ab9dcf1370-tls-certs\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.745516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.745443 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-model-cache\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.745516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.745491 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-tmp-dir\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.745627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.745533 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-home\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.745627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.745559 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcqmm\" (UniqueName: \"kubernetes.io/projected/1f286587-5131-41ed-a704-08ab9dcf1370-kube-api-access-tcqmm\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.845889 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.845868 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-home\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.845899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcqmm\" (UniqueName: \"kubernetes.io/projected/1f286587-5131-41ed-a704-08ab9dcf1370-kube-api-access-tcqmm\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.845937 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-dshm\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.845963 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f286587-5131-41ed-a704-08ab9dcf1370-tls-certs\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.846001 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-model-cache\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846220 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.846023 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-tmp-dir\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846362 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.846338 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-home\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846438 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.846387 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-tmp-dir\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.846518 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.846499 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-model-cache\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.848383 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.848365 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-dshm\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.848560 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.848542 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f286587-5131-41ed-a704-08ab9dcf1370-tls-certs\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.853718 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.853697 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcqmm\" (UniqueName: \"kubernetes.io/projected/1f286587-5131-41ed-a704-08ab9dcf1370-kube-api-access-tcqmm\") pod \"e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:45.918878 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:45.918858 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:46.047616 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:46.047589 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz"] Jul 24 22:38:46.049419 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:38:46.049380 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f286587_5131_41ed_a704_08ab9dcf1370.slice/crio-8c31dfc7e6315c6d491a0e6fad2cbc0e20f15a361ce81f1feda2616ed8af3421 WatchSource:0}: Error finding container 8c31dfc7e6315c6d491a0e6fad2cbc0e20f15a361ce81f1feda2616ed8af3421: Status 404 returned error can't find the container with id 8c31dfc7e6315c6d491a0e6fad2cbc0e20f15a361ce81f1feda2616ed8af3421 Jul 24 22:38:46.494659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:46.494625 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" event={"ID":"1f286587-5131-41ed-a704-08ab9dcf1370","Type":"ContainerStarted","Data":"8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1"} Jul 24 22:38:46.494659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:46.494666 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" event={"ID":"1f286587-5131-41ed-a704-08ab9dcf1370","Type":"ContainerStarted","Data":"8c31dfc7e6315c6d491a0e6fad2cbc0e20f15a361ce81f1feda2616ed8af3421"} Jul 24 22:38:46.512875 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:46.512830 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" podStartSLOduration=1.5128136909999998 podStartE2EDuration="1.512813691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:38:46.510448559 +0000 UTC m=+1423.897266605" watchObservedRunningTime="2026-07-24 22:38:46.512813691 +0000 UTC m=+1423.899631736" Jul 24 22:38:55.919729 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:55.919633 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:55.919729 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:55.919690 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:55.932699 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:55.932673 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:38:56.556472 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:38:56.556441 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:39:25.783376 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:25.783342 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz"] Jul 24 22:39:40.808547 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:40.808510 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" podUID="1f286587-5131-41ed-a704-08ab9dcf1370" containerName="main" containerID="cri-o://8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1" gracePeriod=45 Jul 24 22:39:41.053661 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.053636 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:39:41.120565 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120515 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-tmp-dir\") pod \"1f286587-5131-41ed-a704-08ab9dcf1370\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " Jul 24 22:39:41.120565 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120544 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-dshm\") pod \"1f286587-5131-41ed-a704-08ab9dcf1370\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " Jul 24 22:39:41.120736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120580 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f286587-5131-41ed-a704-08ab9dcf1370-tls-certs\") pod \"1f286587-5131-41ed-a704-08ab9dcf1370\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " Jul 24 22:39:41.120736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120599 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-home\") pod \"1f286587-5131-41ed-a704-08ab9dcf1370\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " Jul 24 22:39:41.120736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120627 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcqmm\" (UniqueName: \"kubernetes.io/projected/1f286587-5131-41ed-a704-08ab9dcf1370-kube-api-access-tcqmm\") pod \"1f286587-5131-41ed-a704-08ab9dcf1370\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " Jul 24 22:39:41.120736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120657 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-model-cache\") pod \"1f286587-5131-41ed-a704-08ab9dcf1370\" (UID: \"1f286587-5131-41ed-a704-08ab9dcf1370\") " Jul 24 22:39:41.120930 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120817 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f286587-5131-41ed-a704-08ab9dcf1370" (UID: "1f286587-5131-41ed-a704-08ab9dcf1370"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:39:41.120930 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120849 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-home" (OuterVolumeSpecName: "home") pod "1f286587-5131-41ed-a704-08ab9dcf1370" (UID: "1f286587-5131-41ed-a704-08ab9dcf1370"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:39:41.121023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120922 2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-tmp-dir\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:39:41.121023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120962 2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-home\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:39:41.121023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.120984 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f286587-5131-41ed-a704-08ab9dcf1370" (UID: "1f286587-5131-41ed-a704-08ab9dcf1370"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:39:41.122829 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.122804 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f286587-5131-41ed-a704-08ab9dcf1370-kube-api-access-tcqmm" (OuterVolumeSpecName: "kube-api-access-tcqmm") pod "1f286587-5131-41ed-a704-08ab9dcf1370" (UID: "1f286587-5131-41ed-a704-08ab9dcf1370"). InnerVolumeSpecName "kube-api-access-tcqmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:39:41.122829 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.122819 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-dshm" (OuterVolumeSpecName: "dshm") pod "1f286587-5131-41ed-a704-08ab9dcf1370" (UID: "1f286587-5131-41ed-a704-08ab9dcf1370"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:39:41.122951 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.122834 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f286587-5131-41ed-a704-08ab9dcf1370-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f286587-5131-41ed-a704-08ab9dcf1370" (UID: "1f286587-5131-41ed-a704-08ab9dcf1370"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 22:39:41.222095 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.222076 2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-dshm\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:39:41.222095 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.222092 2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f286587-5131-41ed-a704-08ab9dcf1370-tls-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:39:41.222234 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.222103 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcqmm\" (UniqueName: \"kubernetes.io/projected/1f286587-5131-41ed-a704-08ab9dcf1370-kube-api-access-tcqmm\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:39:41.222234 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.222111 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f286587-5131-41ed-a704-08ab9dcf1370-model-cache\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:39:41.753737 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.753707 2554 generic.go:356] "Generic (PLEG): container finished" podID="1f286587-5131-41ed-a704-08ab9dcf1370" containerID="8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1" exitCode=0 Jul 24 22:39:41.753880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.753745 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" event={"ID":"1f286587-5131-41ed-a704-08ab9dcf1370","Type":"ContainerDied","Data":"8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1"} Jul 24 22:39:41.753880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.753759 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" Jul 24 22:39:41.753880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.753769 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz" event={"ID":"1f286587-5131-41ed-a704-08ab9dcf1370","Type":"ContainerDied","Data":"8c31dfc7e6315c6d491a0e6fad2cbc0e20f15a361ce81f1feda2616ed8af3421"} Jul 24 22:39:41.753880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.753789 2554 scope.go:117] "RemoveContainer" containerID="8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1" Jul 24 22:39:41.762023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.762007 2554 scope.go:117] "RemoveContainer" containerID="8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1" Jul 24 22:39:41.762314 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:39:41.762293 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1\": container with ID starting with 8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1 not found: ID does not exist" containerID="8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1" Jul 24 22:39:41.762386 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.762325 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1"} err="failed to get container status \"8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1\": rpc error: code = NotFound desc = could not find container \"8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1\": container with ID starting with 8fac974795cfd3ad6dbed000afa38b345c097c48af504fc89398f6de91a103d1 not found: ID does not exist" Jul 24 22:39:41.782759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.782736 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz"] Jul 24 22:39:41.789594 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:41.789574 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-94df6b07-kserve-75655bb547-bfxlz"] Jul 24 22:39:43.175874 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:39:43.175846 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f286587-5131-41ed-a704-08ab9dcf1370" path="/var/lib/kubelet/pods/1f286587-5131-41ed-a704-08ab9dcf1370/volumes" Jul 24 22:43:17.866972 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.866939 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd"] Jul 24 22:43:17.868869 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.867269 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f286587-5131-41ed-a704-08ab9dcf1370" containerName="main" Jul 24 22:43:17.868869 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.867281 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f286587-5131-41ed-a704-08ab9dcf1370" containerName="main" Jul 24 22:43:17.868869 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.867345 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f286587-5131-41ed-a704-08ab9dcf1370" containerName="main" Jul 24 22:43:17.869507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.869491 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.873387 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.873354 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-64bba790-openshift-default-dockercfg-md8jr\"" Jul 24 22:43:17.891091 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.891067 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd"] Jul 24 22:43:17.944751 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtgdn\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-kube-api-access-wtgdn\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.944901 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944753 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-podinfo\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.944901 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944774 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-envoy\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.944901 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944806 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-socket\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.944901 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944821 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-token\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.944901 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944848 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istiod-ca-cert\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.945115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944901 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-credential-socket\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.945115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944934 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-data\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:17.945115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:17.944958 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-certs\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046292 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046255 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istiod-ca-cert\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046479 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046315 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-credential-socket\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046479 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046342 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-data\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046479 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046373 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-certs\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046635 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtgdn\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-kube-api-access-wtgdn\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046690 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046641 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-podinfo\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046690 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046673 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-envoy\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046799 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046703 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-socket\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046799 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046727 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-token\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046799 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046745 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-credential-socket\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046799 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046777 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-data\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.046989 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.046839 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-certs\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.047045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.047027 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istiod-ca-cert\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.047086 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.047029 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-socket\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.049022 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.049001 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-envoy\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.049249 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.049231 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-podinfo\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.055016 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.054989 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-token\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.055295 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.055270 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtgdn\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-kube-api-access-wtgdn\") pod \"e2e-ait-64bba790-openshift-default-79786f8986-7znqd\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.178753 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.178728 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:18.312941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.312913 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd"] Jul 24 22:43:18.315022 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:43:18.314995 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cee7f2_b6d4_4e45_84e7_53ccfca9c1ab.slice/crio-0794c8097d74dc05d5715f98c85acce38344eb6932c6d42a05c5401641c8a579 WatchSource:0}: Error finding container 0794c8097d74dc05d5715f98c85acce38344eb6932c6d42a05c5401641c8a579: Status 404 returned error can't find the container with id 0794c8097d74dc05d5715f98c85acce38344eb6932c6d42a05c5401641c8a579 Jul 24 22:43:18.317008 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.316977 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:43:18.317059 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.317041 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:43:18.317097 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.317068 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:43:18.716871 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.716840 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" event={"ID":"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab","Type":"ContainerStarted","Data":"a4d1077705ac67468b8755560b85002794d62beef0ff1763fe30c8c6910cec7d"} Jul 24 22:43:18.717046 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.716903 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" event={"ID":"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab","Type":"ContainerStarted","Data":"0794c8097d74dc05d5715f98c85acce38344eb6932c6d42a05c5401641c8a579"} Jul 24 22:43:18.738253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:18.738201 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" podStartSLOduration=1.738186857 podStartE2EDuration="1.738186857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:18.736849695 +0000 UTC m=+1696.123667733" watchObservedRunningTime="2026-07-24 22:43:18.738186857 +0000 UTC m=+1696.125004903" Jul 24 22:43:19.179790 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:19.179769 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:19.185713 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:19.185691 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:19.721957 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:19.721925 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:19.722784 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:19.722769 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:20.795340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.795312 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n"] Jul 24 22:43:20.798138 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.798123 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:20.800703 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.800681 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-64bba790-dockercfg-qmbzm\"" Jul 24 22:43:20.800836 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.800681 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-64bba790\"" Jul 24 22:43:20.809006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.808988 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n"] Jul 24 22:43:20.846428 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.846387 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n"] Jul 24 22:43:20.849256 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.849241 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:20.871889 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.871868 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n"] Jul 24 22:43:20.873417 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.873394 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndmh5\" (UniqueName: \"kubernetes.io/projected/f2173353-34eb-481f-80d6-6f1fc0427f1a-kube-api-access-ndmh5\") pod \"payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:20.873527 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.873424 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-plugins-config-volume\") pod \"payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:20.873527 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.873447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrb9r\" (UniqueName: \"kubernetes.io/projected/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-kube-api-access-nrb9r\") pod \"payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:20.873527 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.873480 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2173353-34eb-481f-80d6-6f1fc0427f1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:20.974172 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.974115 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2173353-34eb-481f-80d6-6f1fc0427f1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:20.974318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.974214 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndmh5\" (UniqueName: \"kubernetes.io/projected/f2173353-34eb-481f-80d6-6f1fc0427f1a-kube-api-access-ndmh5\") pod \"payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:20.974318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.974235 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-plugins-config-volume\") pod \"payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:20.974318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.974256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrb9r\" (UniqueName: \"kubernetes.io/projected/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-kube-api-access-nrb9r\") pod \"payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:20.975059 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.975034 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-plugins-config-volume\") pod \"payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:20.975099 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.975034 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2173353-34eb-481f-80d6-6f1fc0427f1a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:20.988904 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:20.988876 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndmh5\" (UniqueName: \"kubernetes.io/projected/f2173353-34eb-481f-80d6-6f1fc0427f1a-kube-api-access-ndmh5\") pod \"payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:21.000270 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.000241 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrb9r\" (UniqueName: \"kubernetes.io/projected/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-kube-api-access-nrb9r\") pod \"payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:21.109087 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.109009 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:21.158498 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.158446 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:21.253753 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.253706 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n"] Jul 24 22:43:21.256625 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:43:21.256526 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2173353_34eb_481f_80d6_6f1fc0427f1a.slice/crio-428acaf5aea267186369c4b02e0bd57e76b6b2068f6f4665deacba59779ae6d1 WatchSource:0}: Error finding container 428acaf5aea267186369c4b02e0bd57e76b6b2068f6f4665deacba59779ae6d1: Status 404 returned error can't find the container with id 428acaf5aea267186369c4b02e0bd57e76b6b2068f6f4665deacba59779ae6d1 Jul 24 22:43:21.308667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.308645 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n"] Jul 24 22:43:21.310771 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:43:21.310745 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35e9b11_c965_452a_ba0f_6cc1e0d0b8bc.slice/crio-515e2779e79b99ae117e3882c81f910a5a3995394e6091e49cad8a2487899b0c WatchSource:0}: Error finding container 515e2779e79b99ae117e3882c81f910a5a3995394e6091e49cad8a2487899b0c: Status 404 returned error can't find the container with id 515e2779e79b99ae117e3882c81f910a5a3995394e6091e49cad8a2487899b0c Jul 24 22:43:21.733899 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.733866 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" event={"ID":"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc","Type":"ContainerStarted","Data":"9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889"} Jul 24 22:43:21.734074 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.733925 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" event={"ID":"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc","Type":"ContainerStarted","Data":"515e2779e79b99ae117e3882c81f910a5a3995394e6091e49cad8a2487899b0c"} Jul 24 22:43:21.734074 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.733954 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:21.735378 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.735350 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" event={"ID":"f2173353-34eb-481f-80d6-6f1fc0427f1a","Type":"ContainerStarted","Data":"32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16"} Jul 24 22:43:21.735497 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.735391 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" event={"ID":"f2173353-34eb-481f-80d6-6f1fc0427f1a","Type":"ContainerStarted","Data":"428acaf5aea267186369c4b02e0bd57e76b6b2068f6f4665deacba59779ae6d1"} Jul 24 22:43:21.735547 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.735511 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:21.749685 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.749644 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" podStartSLOduration=1.7496304139999999 podStartE2EDuration="1.749630414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:21.748456213 +0000 UTC m=+1699.135274265" watchObservedRunningTime="2026-07-24 22:43:21.749630414 +0000 UTC m=+1699.136448473" Jul 24 22:43:21.764297 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:21.764250 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" podStartSLOduration=1.7642366919999999 podStartE2EDuration="1.764236692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:43:21.762689092 +0000 UTC m=+1699.149507138" watchObservedRunningTime="2026-07-24 22:43:21.764236692 +0000 UTC m=+1699.151054738" Jul 24 22:43:31.793871 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:31.793838 2554 generic.go:356] "Generic (PLEG): container finished" podID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerID="a4d1077705ac67468b8755560b85002794d62beef0ff1763fe30c8c6910cec7d" exitCode=0 Jul 24 22:43:31.794279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:31.793907 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" event={"ID":"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab","Type":"ContainerDied","Data":"a4d1077705ac67468b8755560b85002794d62beef0ff1763fe30c8c6910cec7d"} Jul 24 22:43:31.794358 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:31.794344 2554 scope.go:117] "RemoveContainer" containerID="a4d1077705ac67468b8755560b85002794d62beef0ff1763fe30c8c6910cec7d" Jul 24 22:43:31.795469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:31.795438 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:43:31.795600 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:31.795523 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:43:31.795600 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:31.795571 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:43:32.741743 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:32.741711 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:32.741897 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:32.741786 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:32.801502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:32.801463 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" event={"ID":"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab","Type":"ContainerStarted","Data":"df052e6292982576e51c8085fa75b979a3b3afffedaa10924854652da960dbe4"} Jul 24 22:43:33.182383 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:33.182330 2554 patch_prober.go:28] interesting pod/e2e-ait-64bba790-openshift-default-79786f8986-7znqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 22:43:33.182591 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:33.182412 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 22:43:33.184859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:33.184837 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:34.179692 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:34.179656 2554 patch_prober.go:28] interesting pod/e2e-ait-64bba790-openshift-default-79786f8986-7znqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 22:43:34.180081 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:34.179715 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 22:43:34.722373 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:34.722337 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:35.179243 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:35.179209 2554 patch_prober.go:28] interesting pod/e2e-ait-64bba790-openshift-default-79786f8986-7znqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 22:43:35.179441 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:35.179269 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 22:43:36.179361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:36.179330 2554 patch_prober.go:28] interesting pod/e2e-ait-64bba790-openshift-default-79786f8986-7znqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 24 22:43:36.179771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:36.179381 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 24 22:43:37.183895 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:37.183866 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:37.184965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:37.184948 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:43:55.999014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:55.998982 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n"] Jul 24 22:43:55.999490 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:55.999217 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" podUID="d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" containerName="payload-processing" containerID="cri-o://9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889" gracePeriod=30 Jul 24 22:43:56.008475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.008444 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n"] Jul 24 22:43:56.008680 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.008642 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" podUID="f2173353-34eb-481f-80d6-6f1fc0427f1a" containerName="payload-pre-processing" containerID="cri-o://32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16" gracePeriod=30 Jul 24 22:43:56.254991 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.254939 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:56.281284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.281261 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:56.369670 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.369644 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrb9r\" (UniqueName: \"kubernetes.io/projected/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-kube-api-access-nrb9r\") pod \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " Jul 24 22:43:56.369817 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.369703 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndmh5\" (UniqueName: \"kubernetes.io/projected/f2173353-34eb-481f-80d6-6f1fc0427f1a-kube-api-access-ndmh5\") pod \"f2173353-34eb-481f-80d6-6f1fc0427f1a\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " Jul 24 22:43:56.369817 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.369728 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-plugins-config-volume\") pod \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\" (UID: \"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc\") " Jul 24 22:43:56.369817 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.369755 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2173353-34eb-481f-80d6-6f1fc0427f1a-plugins-config-volume\") pod \"f2173353-34eb-481f-80d6-6f1fc0427f1a\" (UID: \"f2173353-34eb-481f-80d6-6f1fc0427f1a\") " Jul 24 22:43:56.370110 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.370081 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" (UID: "d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:43:56.370235 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.370187 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2173353-34eb-481f-80d6-6f1fc0427f1a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2173353-34eb-481f-80d6-6f1fc0427f1a" (UID: "f2173353-34eb-481f-80d6-6f1fc0427f1a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:43:56.372021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.372000 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-kube-api-access-nrb9r" (OuterVolumeSpecName: "kube-api-access-nrb9r") pod "d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" (UID: "d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc"). InnerVolumeSpecName "kube-api-access-nrb9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:43:56.372077 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.372038 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2173353-34eb-481f-80d6-6f1fc0427f1a-kube-api-access-ndmh5" (OuterVolumeSpecName: "kube-api-access-ndmh5") pod "f2173353-34eb-481f-80d6-6f1fc0427f1a" (UID: "f2173353-34eb-481f-80d6-6f1fc0427f1a"). InnerVolumeSpecName "kube-api-access-ndmh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:43:56.470802 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.470779 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrb9r\" (UniqueName: \"kubernetes.io/projected/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-kube-api-access-nrb9r\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:43:56.470802 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.470797 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndmh5\" (UniqueName: \"kubernetes.io/projected/f2173353-34eb-481f-80d6-6f1fc0427f1a-kube-api-access-ndmh5\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:43:56.470938 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.470807 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:43:56.470938 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.470817 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2173353-34eb-481f-80d6-6f1fc0427f1a-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:43:56.923928 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.923893 2554 generic.go:356] "Generic (PLEG): container finished" podID="d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" containerID="9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889" exitCode=0 Jul 24 22:43:56.924099 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.923951 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" Jul 24 22:43:56.924099 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.923989 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" event={"ID":"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc","Type":"ContainerDied","Data":"9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889"} Jul 24 22:43:56.924099 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.924079 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n" event={"ID":"d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc","Type":"ContainerDied","Data":"515e2779e79b99ae117e3882c81f910a5a3995394e6091e49cad8a2487899b0c"} Jul 24 22:43:56.924252 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.924112 2554 scope.go:117] "RemoveContainer" containerID="9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889" Jul 24 22:43:56.925445 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.925420 2554 generic.go:356] "Generic (PLEG): container finished" podID="f2173353-34eb-481f-80d6-6f1fc0427f1a" containerID="32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16" exitCode=0 Jul 24 22:43:56.925511 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.925471 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" Jul 24 22:43:56.925596 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.925514 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" event={"ID":"f2173353-34eb-481f-80d6-6f1fc0427f1a","Type":"ContainerDied","Data":"32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16"} Jul 24 22:43:56.925596 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.925558 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n" event={"ID":"f2173353-34eb-481f-80d6-6f1fc0427f1a","Type":"ContainerDied","Data":"428acaf5aea267186369c4b02e0bd57e76b6b2068f6f4665deacba59779ae6d1"} Jul 24 22:43:56.934633 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.934614 2554 scope.go:117] "RemoveContainer" containerID="9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889" Jul 24 22:43:56.934946 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:43:56.934929 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889\": container with ID starting with 9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889 not found: ID does not exist" containerID="9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889" Jul 24 22:43:56.934990 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.934954 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889"} err="failed to get container status \"9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889\": rpc error: code = NotFound desc = could not find container \"9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889\": container with ID starting with 9267b325f01236fddfe856156c6152aa74ed8a9e0160761071f6e89c70e1a889 not found: ID does not exist" Jul 24 22:43:56.934990 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.934970 2554 scope.go:117] "RemoveContainer" containerID="32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16" Jul 24 22:43:56.943901 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.943884 2554 scope.go:117] "RemoveContainer" containerID="32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16" Jul 24 22:43:56.944211 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:43:56.944189 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16\": container with ID starting with 32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16 not found: ID does not exist" containerID="32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16" Jul 24 22:43:56.944274 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.944220 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16"} err="failed to get container status \"32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16\": rpc error: code = NotFound desc = could not find container \"32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16\": container with ID starting with 32a7263a22982f0493bd642c7f17e5f30e9d2d96fa03937a9b3c1e223fec3f16 not found: ID does not exist" Jul 24 22:43:56.949553 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.949529 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n"] Jul 24 22:43:56.953572 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.953544 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64bba790-557d95b6fb-n4t7n"] Jul 24 22:43:56.962465 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.962445 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n"] Jul 24 22:43:56.966646 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:56.966622 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-64bba790-6cc6b5c9bb-mk48n"] Jul 24 22:43:57.177333 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:57.177249 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" path="/var/lib/kubelet/pods/d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc/volumes" Jul 24 22:43:57.177777 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:43:57.177735 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2173353-34eb-481f-80d6-6f1fc0427f1a" path="/var/lib/kubelet/pods/f2173353-34eb-481f-80d6-6f1fc0427f1a/volumes" Jul 24 22:44:00.752012 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:00.751978 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd"] Jul 24 22:44:00.752492 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:00.752214 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" containerID="cri-o://df052e6292982576e51c8085fa75b979a3b3afffedaa10924854652da960dbe4" gracePeriod=30 Jul 24 22:44:01.727167 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727113 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt"] Jul 24 22:44:01.727513 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727493 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2173353-34eb-481f-80d6-6f1fc0427f1a" containerName="payload-pre-processing" Jul 24 22:44:01.727592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727517 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2173353-34eb-481f-80d6-6f1fc0427f1a" containerName="payload-pre-processing" Jul 24 22:44:01.727592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727532 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" containerName="payload-processing" Jul 24 22:44:01.727592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727538 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" containerName="payload-processing" Jul 24 22:44:01.727592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727591 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2173353-34eb-481f-80d6-6f1fc0427f1a" containerName="payload-pre-processing" Jul 24 22:44:01.727760 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.727599 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d35e9b11-c965-452a-ba0f-6cc1e0d0b8bc" containerName="payload-processing" Jul 24 22:44:01.732138 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.732118 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.734815 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.734798 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1bab8920-gw-openshift-default-dockercfg-k8tcj\"" Jul 24 22:44:01.742421 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.742401 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt"] Jul 24 22:44:01.910669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910633 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-envoy\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910675 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f709883-78b7-4600-80d0-75a8fd8ee368-istiod-ca-cert\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910698 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-data\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910750 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-socket\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910796 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-kube-api-access-s8fhh\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910848 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-certs\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910946 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-podinfo\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.910984 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-credential-socket\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:01.911134 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:01.911046 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-token\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012229 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012136 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-credential-socket\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012229 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012209 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-token\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012240 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-envoy\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012266 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f709883-78b7-4600-80d0-75a8fd8ee368-istiod-ca-cert\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012295 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-data\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012324 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-socket\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012347 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-kube-api-access-s8fhh\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012391 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-certs\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012469 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-podinfo\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012565 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-credential-socket\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012811 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012740 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-data\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012811 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012751 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-socket\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.012811 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.012788 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-certs\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.013022 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.013000 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f709883-78b7-4600-80d0-75a8fd8ee368-istiod-ca-cert\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.014619 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.014595 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-envoy\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.015007 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.014991 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-podinfo\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.026632 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.026603 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-kube-api-access-s8fhh\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.026757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.026733 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-token\") pod \"e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.043620 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.043596 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:02.188720 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.188700 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt"] Jul 24 22:44:02.190083 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:44:02.190060 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f709883_78b7_4600_80d0_75a8fd8ee368.slice/crio-6d9a8974376a329d363fa7758222cd275520b4dc1f9846cafafbe972d02c23ac WatchSource:0}: Error finding container 6d9a8974376a329d363fa7758222cd275520b4dc1f9846cafafbe972d02c23ac: Status 404 returned error can't find the container with id 6d9a8974376a329d363fa7758222cd275520b4dc1f9846cafafbe972d02c23ac Jul 24 22:44:02.192075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.192041 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:02.192118 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.192110 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:02.192166 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.192137 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:02.958777 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.958744 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" event={"ID":"1f709883-78b7-4600-80d0-75a8fd8ee368","Type":"ContainerStarted","Data":"727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b"} Jul 24 22:44:02.959176 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.958791 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" event={"ID":"1f709883-78b7-4600-80d0-75a8fd8ee368","Type":"ContainerStarted","Data":"6d9a8974376a329d363fa7758222cd275520b4dc1f9846cafafbe972d02c23ac"} Jul 24 22:44:02.981745 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:02.981696 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" podStartSLOduration=1.981680376 podStartE2EDuration="1.981680376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:02.97921687 +0000 UTC m=+1740.366034920" watchObservedRunningTime="2026-07-24 22:44:02.981680376 +0000 UTC m=+1740.368498421" Jul 24 22:44:03.043955 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:03.043931 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:03.048767 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:03.048747 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:03.964285 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:03.964253 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:03.965393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:03.965369 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:05.093517 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.093484 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg"] Jul 24 22:44:05.098211 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.098192 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.102071 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.102049 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-1bab8920-dockercfg-dv7x5\"" Jul 24 22:44:05.102185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.102072 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-1bab8920\"" Jul 24 22:44:05.106925 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.106906 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk"] Jul 24 22:44:05.110487 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.110471 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.110997 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.110980 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg"] Jul 24 22:44:05.123785 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.123760 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk"] Jul 24 22:44:05.138921 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.138886 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.139021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.138930 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptss5\" (UniqueName: \"kubernetes.io/projected/6a40f745-9ac7-4189-b487-732e52845b57-kube-api-access-ptss5\") pod \"payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.139021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.139007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a40f745-9ac7-4189-b487-732e52845b57-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.139180 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.139063 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dfs6\" (UniqueName: \"kubernetes.io/projected/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-kube-api-access-2dfs6\") pod \"payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.239833 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.239796 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a40f745-9ac7-4189-b487-732e52845b57-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.240044 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.239844 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dfs6\" (UniqueName: \"kubernetes.io/projected/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-kube-api-access-2dfs6\") pod \"payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.240044 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.239867 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.240044 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.239925 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptss5\" (UniqueName: \"kubernetes.io/projected/6a40f745-9ac7-4189-b487-732e52845b57-kube-api-access-ptss5\") pod \"payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.240723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.240700 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a40f745-9ac7-4189-b487-732e52845b57-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.240804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.240785 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.248690 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.248656 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptss5\" (UniqueName: \"kubernetes.io/projected/6a40f745-9ac7-4189-b487-732e52845b57-kube-api-access-ptss5\") pod \"payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.248828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.248659 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dfs6\" (UniqueName: \"kubernetes.io/projected/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-kube-api-access-2dfs6\") pod \"payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.408778 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.408748 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.420098 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.420070 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.554853 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.554606 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg"] Jul 24 22:44:05.557259 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:44:05.557229 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a40f745_9ac7_4189_b487_732e52845b57.slice/crio-a37b1e2982995cd49ce640aae0a4d8a2e53b5d0c1bdf0f85a82317669780a8df WatchSource:0}: Error finding container a37b1e2982995cd49ce640aae0a4d8a2e53b5d0c1bdf0f85a82317669780a8df: Status 404 returned error can't find the container with id a37b1e2982995cd49ce640aae0a4d8a2e53b5d0c1bdf0f85a82317669780a8df Jul 24 22:44:05.577973 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.577950 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk"] Jul 24 22:44:05.579252 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:44:05.579229 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67daeb4f_4af2_4a1b_b6ea_8e90e2780cb2.slice/crio-08bf8d2426237de2d044957e5055194626165f85a778148a552b485b942ba8ec WatchSource:0}: Error finding container 08bf8d2426237de2d044957e5055194626165f85a778148a552b485b942ba8ec: Status 404 returned error can't find the container with id 08bf8d2426237de2d044957e5055194626165f85a778148a552b485b942ba8ec Jul 24 22:44:05.975135 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.975098 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" event={"ID":"6a40f745-9ac7-4189-b487-732e52845b57","Type":"ContainerStarted","Data":"d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745"} Jul 24 22:44:05.975303 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.975190 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" event={"ID":"6a40f745-9ac7-4189-b487-732e52845b57","Type":"ContainerStarted","Data":"a37b1e2982995cd49ce640aae0a4d8a2e53b5d0c1bdf0f85a82317669780a8df"} Jul 24 22:44:05.975303 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.975221 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:05.976872 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.976843 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" event={"ID":"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2","Type":"ContainerStarted","Data":"cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a"} Jul 24 22:44:05.977044 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.976893 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" event={"ID":"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2","Type":"ContainerStarted","Data":"08bf8d2426237de2d044957e5055194626165f85a778148a552b485b942ba8ec"} Jul 24 22:44:05.977044 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.976954 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:05.979027 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.979006 2554 generic.go:356] "Generic (PLEG): container finished" podID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerID="df052e6292982576e51c8085fa75b979a3b3afffedaa10924854652da960dbe4" exitCode=0 Jul 24 22:44:05.979145 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.979033 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" event={"ID":"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab","Type":"ContainerDied","Data":"df052e6292982576e51c8085fa75b979a3b3afffedaa10924854652da960dbe4"} Jul 24 22:44:05.979145 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:05.979080 2554 scope.go:117] "RemoveContainer" containerID="a4d1077705ac67468b8755560b85002794d62beef0ff1763fe30c8c6910cec7d" Jul 24 22:44:06.006031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.006013 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:44:06.007544 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.007506 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" podStartSLOduration=1.007492906 podStartE2EDuration="1.007492906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:05.992131487 +0000 UTC m=+1743.378949535" watchObservedRunningTime="2026-07-24 22:44:06.007492906 +0000 UTC m=+1743.394310951" Jul 24 22:44:06.007633 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.007577 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" podStartSLOduration=1.007571485 podStartE2EDuration="1.007571485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:06.006928582 +0000 UTC m=+1743.393746628" watchObservedRunningTime="2026-07-24 22:44:06.007571485 +0000 UTC m=+1743.394389553" Jul 24 22:44:06.046835 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046773 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-podinfo\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.046835 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046807 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-socket\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.046835 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046826 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-certs\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046858 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istiod-ca-cert\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046900 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtgdn\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-kube-api-access-wtgdn\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046923 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-token\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046940 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-data\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.046967 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-envoy\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047013 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-credential-socket\") pod \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\" (UID: \"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab\") " Jul 24 22:44:06.047346 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047141 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:06.047346 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047167 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:06.047453 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047370 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.047453 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047388 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:44:06.047453 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047395 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.047600 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047476 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:06.048211 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.047878 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:06.049340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.049311 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:44:06.049426 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.049313 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:06.049572 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.049554 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-kube-api-access-wtgdn" (OuterVolumeSpecName: "kube-api-access-wtgdn") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "kube-api-access-wtgdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:06.049954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.049936 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" (UID: "a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:06.148828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148797 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.148828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148827 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtgdn\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-kube-api-access-wtgdn\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.149265 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148840 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.149265 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148852 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.149265 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148862 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.149265 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148873 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.149265 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.148885 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:06.986730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.986703 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" Jul 24 22:44:06.986896 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.986742 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd" event={"ID":"a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab","Type":"ContainerDied","Data":"0794c8097d74dc05d5715f98c85acce38344eb6932c6d42a05c5401641c8a579"} Jul 24 22:44:06.986896 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:06.986794 2554 scope.go:117] "RemoveContainer" containerID="df052e6292982576e51c8085fa75b979a3b3afffedaa10924854652da960dbe4" Jul 24 22:44:07.008735 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:07.008711 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd"] Jul 24 22:44:07.012250 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:07.012231 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-64bba790-openshift-default-79786f8986-7znqd"] Jul 24 22:44:07.177741 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:07.177714 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" path="/var/lib/kubelet/pods/a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab/volumes" Jul 24 22:44:16.988835 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:16.988799 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:16.989538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:16.989513 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:19.325639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:19.325597 2554 patch_prober.go:28] interesting pod/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": read tcp 10.134.0.2:49178->10.134.0.37:15021: read: connection reset by peer" start-of-body= Jul 24 22:44:19.325985 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:19.325660 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": read tcp 10.134.0.2:49178->10.134.0.37:15021: read: connection reset by peer" Jul 24 22:44:20.056657 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:20.056624 2554 generic.go:356] "Generic (PLEG): container finished" podID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerID="727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b" exitCode=0 Jul 24 22:44:20.056847 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:20.056696 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" event={"ID":"1f709883-78b7-4600-80d0-75a8fd8ee368","Type":"ContainerDied","Data":"727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b"} Jul 24 22:44:20.057099 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:20.057085 2554 scope.go:117] "RemoveContainer" containerID="727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b" Jul 24 22:44:20.058124 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:20.058087 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:20.058238 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:20.058189 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:20.058277 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:20.058241 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:21.066854 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:21.066813 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" event={"ID":"1f709883-78b7-4600-80d0-75a8fd8ee368","Type":"ContainerStarted","Data":"cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27"} Jul 24 22:44:22.044060 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:22.044025 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:22.044279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:22.044214 2554 patch_prober.go:28] interesting pod/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 24 22:44:22.044279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:22.044263 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 24 22:44:23.045037 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:23.045003 2554 patch_prober.go:28] interesting pod/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 24 22:44:23.045447 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:23.045054 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 24 22:44:24.045049 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:24.045014 2554 patch_prober.go:28] interesting pod/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 24 22:44:24.045461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:24.045072 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 24 22:44:25.048506 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:25.048481 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:25.048860 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:25.048742 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:25.049549 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:25.049529 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:39.759679 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:39.759647 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg"] Jul 24 22:44:39.760252 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:39.759896 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" podUID="6a40f745-9ac7-4189-b487-732e52845b57" containerName="payload-processing" containerID="cri-o://d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745" gracePeriod=30 Jul 24 22:44:39.764746 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:39.764694 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk"] Jul 24 22:44:39.764911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:39.764890 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" podUID="67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" containerName="payload-pre-processing" containerID="cri-o://cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a" gracePeriod=30 Jul 24 22:44:40.032052 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.032023 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:40.057040 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.057017 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:40.127979 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.127942 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a40f745-9ac7-4189-b487-732e52845b57-plugins-config-volume\") pod \"6a40f745-9ac7-4189-b487-732e52845b57\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " Jul 24 22:44:40.128207 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.127998 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptss5\" (UniqueName: \"kubernetes.io/projected/6a40f745-9ac7-4189-b487-732e52845b57-kube-api-access-ptss5\") pod \"6a40f745-9ac7-4189-b487-732e52845b57\" (UID: \"6a40f745-9ac7-4189-b487-732e52845b57\") " Jul 24 22:44:40.128207 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.128032 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dfs6\" (UniqueName: \"kubernetes.io/projected/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-kube-api-access-2dfs6\") pod \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " Jul 24 22:44:40.128207 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.128084 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-plugins-config-volume\") pod \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\" (UID: \"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2\") " Jul 24 22:44:40.128494 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.128460 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a40f745-9ac7-4189-b487-732e52845b57-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6a40f745-9ac7-4189-b487-732e52845b57" (UID: "6a40f745-9ac7-4189-b487-732e52845b57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:44:40.128581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.128527 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" (UID: "67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:44:40.130446 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.130420 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a40f745-9ac7-4189-b487-732e52845b57-kube-api-access-ptss5" (OuterVolumeSpecName: "kube-api-access-ptss5") pod "6a40f745-9ac7-4189-b487-732e52845b57" (UID: "6a40f745-9ac7-4189-b487-732e52845b57"). InnerVolumeSpecName "kube-api-access-ptss5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:40.130540 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.130445 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-kube-api-access-2dfs6" (OuterVolumeSpecName: "kube-api-access-2dfs6") pod "67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" (UID: "67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2"). InnerVolumeSpecName "kube-api-access-2dfs6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:40.167311 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.167274 2554 generic.go:356] "Generic (PLEG): container finished" podID="6a40f745-9ac7-4189-b487-732e52845b57" containerID="d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745" exitCode=0 Jul 24 22:44:40.167502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.167343 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" Jul 24 22:44:40.167502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.167357 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" event={"ID":"6a40f745-9ac7-4189-b487-732e52845b57","Type":"ContainerDied","Data":"d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745"} Jul 24 22:44:40.167502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.167444 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg" event={"ID":"6a40f745-9ac7-4189-b487-732e52845b57","Type":"ContainerDied","Data":"a37b1e2982995cd49ce640aae0a4d8a2e53b5d0c1bdf0f85a82317669780a8df"} Jul 24 22:44:40.167502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.167474 2554 scope.go:117] "RemoveContainer" containerID="d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745" Jul 24 22:44:40.168789 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.168766 2554 generic.go:356] "Generic (PLEG): container finished" podID="67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" containerID="cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a" exitCode=0 Jul 24 22:44:40.168889 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.168807 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" event={"ID":"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2","Type":"ContainerDied","Data":"cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a"} Jul 24 22:44:40.168889 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.168832 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" Jul 24 22:44:40.168994 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.168837 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk" event={"ID":"67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2","Type":"ContainerDied","Data":"08bf8d2426237de2d044957e5055194626165f85a778148a552b485b942ba8ec"} Jul 24 22:44:40.177782 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.177764 2554 scope.go:117] "RemoveContainer" containerID="d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745" Jul 24 22:44:40.178080 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:44:40.178062 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745\": container with ID starting with d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745 not found: ID does not exist" containerID="d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745" Jul 24 22:44:40.178189 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.178090 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745"} err="failed to get container status \"d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745\": rpc error: code = NotFound desc = could not find container \"d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745\": container with ID starting with d03b59c8c42441d12b16b9534d775991390d9d99f99b5d2b865c9d051a50a745 not found: ID does not exist" Jul 24 22:44:40.178189 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.178111 2554 scope.go:117] "RemoveContainer" containerID="cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a" Jul 24 22:44:40.186615 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.186596 2554 scope.go:117] "RemoveContainer" containerID="cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a" Jul 24 22:44:40.186889 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:44:40.186871 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a\": container with ID starting with cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a not found: ID does not exist" containerID="cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a" Jul 24 22:44:40.186954 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.186896 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a"} err="failed to get container status \"cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a\": rpc error: code = NotFound desc = could not find container \"cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a\": container with ID starting with cb20b56d597b33ad37f045fe0692d50bd1e0811e5abca6f75698b9722768642a not found: ID does not exist" Jul 24 22:44:40.192514 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.192489 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg"] Jul 24 22:44:40.196321 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.196296 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1bab8920-6fc8bcd485-hw6zg"] Jul 24 22:44:40.205175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.205131 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk"] Jul 24 22:44:40.208474 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.208449 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1bab8920-6d6cfc694d-vvwvk"] Jul 24 22:44:40.228848 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.228816 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptss5\" (UniqueName: \"kubernetes.io/projected/6a40f745-9ac7-4189-b487-732e52845b57-kube-api-access-ptss5\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:40.228848 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.228843 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dfs6\" (UniqueName: \"kubernetes.io/projected/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-kube-api-access-2dfs6\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:40.228848 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.228855 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:40.229046 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:40.228864 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6a40f745-9ac7-4189-b487-732e52845b57-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:41.175585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:41.175551 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" path="/var/lib/kubelet/pods/67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2/volumes" Jul 24 22:44:41.176082 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:41.176058 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a40f745-9ac7-4189-b487-732e52845b57" path="/var/lib/kubelet/pods/6a40f745-9ac7-4189-b487-732e52845b57/volumes" Jul 24 22:44:44.600536 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:44.600505 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt"] Jul 24 22:44:44.600956 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:44.600736 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" containerID="cri-o://cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27" gracePeriod=30 Jul 24 22:44:45.358006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.357975 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh"] Jul 24 22:44:45.358422 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358404 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" Jul 24 22:44:45.358422 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358424 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358434 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" containerName="payload-pre-processing" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358440 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" containerName="payload-pre-processing" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358456 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a40f745-9ac7-4189-b487-732e52845b57" containerName="payload-processing" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358461 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a40f745-9ac7-4189-b487-732e52845b57" containerName="payload-processing" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358512 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a40f745-9ac7-4189-b487-732e52845b57" containerName="payload-processing" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358519 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358526 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" Jul 24 22:44:45.358551 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.358533 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="67daeb4f-4af2-4a1b-b6ea-8e90e2780cb2" containerName="payload-pre-processing" Jul 24 22:44:45.363339 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.363314 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.365918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.365901 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2db675aa-openshift-default-dockercfg-q722b\"" Jul 24 22:44:45.372875 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.372842 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh"] Jul 24 22:44:45.474211 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474141 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db6jc\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-kube-api-access-db6jc\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474367 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474229 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-envoy\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474367 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474269 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-istio-token\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474367 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474318 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-certs\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474492 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474367 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-socket\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474492 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474392 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5281da-5fae-4581-95c2-251c35c766ed-istiod-ca-cert\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474492 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474412 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5281da-5fae-4581-95c2-251c35c766ed-istio-podinfo\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474492 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-credential-socket\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.474634 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.474503 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-data\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.575624 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575594 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db6jc\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-kube-api-access-db6jc\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.575804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575638 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-envoy\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.575804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575670 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-istio-token\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.575804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575708 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-certs\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.575804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575749 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-socket\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.575804 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575784 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5281da-5fae-4581-95c2-251c35c766ed-istiod-ca-cert\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575819 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5281da-5fae-4581-95c2-251c35c766ed-istio-podinfo\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575867 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-credential-socket\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.575896 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-data\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576334 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.576306 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-socket\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576563 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.576540 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-certs\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576688 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.576669 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-credential-socket\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576753 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.576728 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-data\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.576815 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.576773 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5281da-5fae-4581-95c2-251c35c766ed-istiod-ca-cert\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.578560 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.578537 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-envoy\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.578659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.578641 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5281da-5fae-4581-95c2-251c35c766ed-istio-podinfo\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.583639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.583613 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-istio-token\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.583826 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.583809 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db6jc\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-kube-api-access-db6jc\") pod \"e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.673837 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.673813 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:45.806559 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.806532 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh"] Jul 24 22:44:45.808205 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:44:45.808176 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5281da_5fae_4581_95c2_251c35c766ed.slice/crio-1f918858ab7de394a92a41b8dfe39ad982d7ae9c3dbbf935271a9233bf99f8da WatchSource:0}: Error finding container 1f918858ab7de394a92a41b8dfe39ad982d7ae9c3dbbf935271a9233bf99f8da: Status 404 returned error can't find the container with id 1f918858ab7de394a92a41b8dfe39ad982d7ae9c3dbbf935271a9233bf99f8da Jul 24 22:44:45.810379 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.810344 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:45.810487 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.810431 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:45.810487 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:45.810478 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:44:46.201494 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:46.201457 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" event={"ID":"8c5281da-5fae-4581-95c2-251c35c766ed","Type":"ContainerStarted","Data":"78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06"} Jul 24 22:44:46.201653 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:46.201514 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" event={"ID":"8c5281da-5fae-4581-95c2-251c35c766ed","Type":"ContainerStarted","Data":"1f918858ab7de394a92a41b8dfe39ad982d7ae9c3dbbf935271a9233bf99f8da"} Jul 24 22:44:46.223990 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:46.223939 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podStartSLOduration=1.223924533 podStartE2EDuration="1.223924533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:46.222050855 +0000 UTC m=+1783.608868902" watchObservedRunningTime="2026-07-24 22:44:46.223924533 +0000 UTC m=+1783.610742627" Jul 24 22:44:46.674939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:46.674899 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:46.680098 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:46.680069 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:47.207002 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:47.206964 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:47.208075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:47.208053 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:44:48.209320 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.209291 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc"] Jul 24 22:44:48.209759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.209725 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" Jul 24 22:44:48.209759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.209742 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1cee7f2-b6d4-4e45-84e7-53ccfca9c1ab" containerName="istio-proxy" Jul 24 22:44:48.213747 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.213724 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.216668 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.216644 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-2db675aa\"" Jul 24 22:44:48.216797 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.216726 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-2db675aa-dockercfg-n7rtm\"" Jul 24 22:44:48.224536 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.224516 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc"] Jul 24 22:44:48.248827 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.248802 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq"] Jul 24 22:44:48.252618 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.252598 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.262138 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.262114 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq"] Jul 24 22:44:48.298724 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.298688 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq598\" (UniqueName: \"kubernetes.io/projected/d9114ba2-53e6-487f-a528-0c0c81a9df3f-kube-api-access-xq598\") pod \"payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.298892 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.298853 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9114ba2-53e6-487f-a528-0c0c81a9df3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.399667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.399634 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6vf7\" (UniqueName: \"kubernetes.io/projected/b49ecb3d-9721-43b6-ae31-6b239b60da03-kube-api-access-l6vf7\") pod \"payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.399667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.399677 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b49ecb3d-9721-43b6-ae31-6b239b60da03-plugins-config-volume\") pod \"payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.399925 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.399712 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9114ba2-53e6-487f-a528-0c0c81a9df3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.399925 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.399836 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq598\" (UniqueName: \"kubernetes.io/projected/d9114ba2-53e6-487f-a528-0c0c81a9df3f-kube-api-access-xq598\") pod \"payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.400474 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.400456 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9114ba2-53e6-487f-a528-0c0c81a9df3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.408307 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.408282 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq598\" (UniqueName: \"kubernetes.io/projected/d9114ba2-53e6-487f-a528-0c0c81a9df3f-kube-api-access-xq598\") pod \"payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.500726 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.500663 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6vf7\" (UniqueName: \"kubernetes.io/projected/b49ecb3d-9721-43b6-ae31-6b239b60da03-kube-api-access-l6vf7\") pod \"payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.500726 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.500702 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b49ecb3d-9721-43b6-ae31-6b239b60da03-plugins-config-volume\") pod \"payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.501375 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.501355 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b49ecb3d-9721-43b6-ae31-6b239b60da03-plugins-config-volume\") pod \"payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.508538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.508517 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6vf7\" (UniqueName: \"kubernetes.io/projected/b49ecb3d-9721-43b6-ae31-6b239b60da03-kube-api-access-l6vf7\") pod \"payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.525214 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.525193 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:48.566621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.566596 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:48.663824 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.663782 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc"] Jul 24 22:44:48.667259 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:44:48.667230 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9114ba2_53e6_487f_a528_0c0c81a9df3f.slice/crio-1cda5bda78560648aed104caec6a4468bcf0485981c7075f55653bf86ed804a4 WatchSource:0}: Error finding container 1cda5bda78560648aed104caec6a4468bcf0485981c7075f55653bf86ed804a4: Status 404 returned error can't find the container with id 1cda5bda78560648aed104caec6a4468bcf0485981c7075f55653bf86ed804a4 Jul 24 22:44:48.718127 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:48.718101 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq"] Jul 24 22:44:48.720104 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:44:48.720081 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb49ecb3d_9721_43b6_ae31_6b239b60da03.slice/crio-66b7570b592a71f807eb7e8977fd158c9fa9f47c4ed05e044e534147251e1338 WatchSource:0}: Error finding container 66b7570b592a71f807eb7e8977fd158c9fa9f47c4ed05e044e534147251e1338: Status 404 returned error can't find the container with id 66b7570b592a71f807eb7e8977fd158c9fa9f47c4ed05e044e534147251e1338 Jul 24 22:44:49.217672 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.217633 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" event={"ID":"b49ecb3d-9721-43b6-ae31-6b239b60da03","Type":"ContainerStarted","Data":"e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f"} Jul 24 22:44:49.218112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.217702 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:44:49.218112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.217729 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" event={"ID":"b49ecb3d-9721-43b6-ae31-6b239b60da03","Type":"ContainerStarted","Data":"66b7570b592a71f807eb7e8977fd158c9fa9f47c4ed05e044e534147251e1338"} Jul 24 22:44:49.219271 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.219246 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" event={"ID":"d9114ba2-53e6-487f-a528-0c0c81a9df3f","Type":"ContainerStarted","Data":"1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92"} Jul 24 22:44:49.219409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.219299 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" event={"ID":"d9114ba2-53e6-487f-a528-0c0c81a9df3f","Type":"ContainerStarted","Data":"1cda5bda78560648aed104caec6a4468bcf0485981c7075f55653bf86ed804a4"} Jul 24 22:44:49.219409 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.219327 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:44:49.235533 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.235485 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" podStartSLOduration=1.23547252 podStartE2EDuration="1.23547252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:49.233518754 +0000 UTC m=+1786.620336800" watchObservedRunningTime="2026-07-24 22:44:49.23547252 +0000 UTC m=+1786.622290566" Jul 24 22:44:49.249362 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.249321 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" podStartSLOduration=1.249310052 podStartE2EDuration="1.249310052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:44:49.24745118 +0000 UTC m=+1786.634269230" watchObservedRunningTime="2026-07-24 22:44:49.249310052 +0000 UTC m=+1786.636128167" Jul 24 22:44:49.851136 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:49.851110 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:50.015315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015287 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-credential-socket\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015324 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-token\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015347 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-certs\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015371 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-socket\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015410 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-envoy\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015425 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f709883-78b7-4600-80d0-75a8fd8ee368-istiod-ca-cert\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015452 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-kube-api-access-s8fhh\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.015510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015482 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-podinfo\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015616 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015642 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015766 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-data\") pod \"1f709883-78b7-4600-80d0-75a8fd8ee368\" (UID: \"1f709883-78b7-4600-80d0-75a8fd8ee368\") " Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015836 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.015879 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f709883-78b7-4600-80d0-75a8fd8ee368-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.016095 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.016117 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f709883-78b7-4600-80d0-75a8fd8ee368-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.016180 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.016538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.016267 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.017013 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.016632 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:50.017816 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.017787 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-kube-api-access-s8fhh" (OuterVolumeSpecName: "kube-api-access-s8fhh") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "kube-api-access-s8fhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:50.018001 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.017981 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:44:50.018255 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.018235 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:44:50.018325 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.018289 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f709883-78b7-4600-80d0-75a8fd8ee368" (UID: "1f709883-78b7-4600-80d0-75a8fd8ee368"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:44:50.117541 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.117517 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8fhh\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-kube-api-access-s8fhh\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.117541 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.117538 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.117541 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.117547 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.117723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.117556 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.117723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.117563 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f709883-78b7-4600-80d0-75a8fd8ee368-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:44:50.225465 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.225435 2554 generic.go:356] "Generic (PLEG): container finished" podID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerID="cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27" exitCode=0 Jul 24 22:44:50.225851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.225488 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" Jul 24 22:44:50.225851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.225521 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" event={"ID":"1f709883-78b7-4600-80d0-75a8fd8ee368","Type":"ContainerDied","Data":"cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27"} Jul 24 22:44:50.225851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.225570 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt" event={"ID":"1f709883-78b7-4600-80d0-75a8fd8ee368","Type":"ContainerDied","Data":"6d9a8974376a329d363fa7758222cd275520b4dc1f9846cafafbe972d02c23ac"} Jul 24 22:44:50.225851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.225593 2554 scope.go:117] "RemoveContainer" containerID="cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27" Jul 24 22:44:50.235522 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.235507 2554 scope.go:117] "RemoveContainer" containerID="727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b" Jul 24 22:44:50.244137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.244078 2554 scope.go:117] "RemoveContainer" containerID="cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27" Jul 24 22:44:50.244511 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:44:50.244473 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27\": container with ID starting with cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27 not found: ID does not exist" containerID="cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27" Jul 24 22:44:50.244652 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.244513 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27"} err="failed to get container status \"cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27\": rpc error: code = NotFound desc = could not find container \"cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27\": container with ID starting with cd34585c03afb3de5518fe6a086b4408f486b6f265ac383559f8d89da06e8d27 not found: ID does not exist" Jul 24 22:44:50.244652 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.244551 2554 scope.go:117] "RemoveContainer" containerID="727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b" Jul 24 22:44:50.245066 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:44:50.245012 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b\": container with ID starting with 727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b not found: ID does not exist" containerID="727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b" Jul 24 22:44:50.245066 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.245049 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b"} err="failed to get container status \"727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b\": rpc error: code = NotFound desc = could not find container \"727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b\": container with ID starting with 727f034ffdff5e1349a34bb743a85a8178794aa1dadf0f293fdf454ccca40f8b not found: ID does not exist" Jul 24 22:44:50.247340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.247319 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt"] Jul 24 22:44:50.251120 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:50.251098 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1bab8920-gw-openshift-default-6699b59bc6-lwkmt"] Jul 24 22:44:51.176053 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:44:51.176014 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" path="/var/lib/kubelet/pods/1f709883-78b7-4600-80d0-75a8fd8ee368/volumes" Jul 24 22:45:00.227864 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:00.227780 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:45:00.228321 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:00.227888 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:45:03.111096 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.110617 2554 patch_prober.go:28] interesting pod/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:35006->10.134.0.40:15021: read: connection reset by peer" start-of-body= Jul 24 22:45:03.111096 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.110678 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:35006->10.134.0.40:15021: read: connection reset by peer" Jul 24 22:45:03.302993 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.302942 2554 generic.go:356] "Generic (PLEG): container finished" podID="8c5281da-5fae-4581-95c2-251c35c766ed" containerID="78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06" exitCode=0 Jul 24 22:45:03.303094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.303012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" event={"ID":"8c5281da-5fae-4581-95c2-251c35c766ed","Type":"ContainerDied","Data":"78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06"} Jul 24 22:45:03.303362 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.303348 2554 scope.go:117] "RemoveContainer" containerID="78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06" Jul 24 22:45:03.304210 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.304179 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:03.304304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.304249 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:03.304304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:03.304279 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:04.309040 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:04.309005 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" event={"ID":"8c5281da-5fae-4581-95c2-251c35c766ed","Type":"ContainerStarted","Data":"95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78"} Jul 24 22:45:04.675198 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:04.675129 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:45:04.675414 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:04.675269 2554 patch_prober.go:28] interesting pod/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 22:45:04.675414 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:04.675321 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 22:45:05.675091 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:05.675051 2554 patch_prober.go:28] interesting pod/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 22:45:05.675520 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:05.675140 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 22:45:06.675075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:06.675038 2554 patch_prober.go:28] interesting pod/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 22:45:06.675291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:06.675089 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 22:45:07.674597 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:07.674561 2554 patch_prober.go:28] interesting pod/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 24 22:45:07.675257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:07.675225 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 24 22:45:08.677611 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:08.677582 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:45:08.677985 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:08.677896 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:45:08.678543 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:08.678528 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:45:23.667535 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.667502 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq"] Jul 24 22:45:23.667974 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.667771 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" podUID="b49ecb3d-9721-43b6-ae31-6b239b60da03" containerName="payload-processing" containerID="cri-o://e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f" gracePeriod=30 Jul 24 22:45:23.672660 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.672633 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc"] Jul 24 22:45:23.673066 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.673011 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" podUID="d9114ba2-53e6-487f-a528-0c0c81a9df3f" containerName="payload-pre-processing" containerID="cri-o://1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92" gracePeriod=30 Jul 24 22:45:23.939943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.939918 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:45:23.952903 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.952887 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:45:23.958280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.958226 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b49ecb3d-9721-43b6-ae31-6b239b60da03-plugins-config-volume\") pod \"b49ecb3d-9721-43b6-ae31-6b239b60da03\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " Jul 24 22:45:23.958397 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.958335 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6vf7\" (UniqueName: \"kubernetes.io/projected/b49ecb3d-9721-43b6-ae31-6b239b60da03-kube-api-access-l6vf7\") pod \"b49ecb3d-9721-43b6-ae31-6b239b60da03\" (UID: \"b49ecb3d-9721-43b6-ae31-6b239b60da03\") " Jul 24 22:45:23.958699 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.958664 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49ecb3d-9721-43b6-ae31-6b239b60da03-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b49ecb3d-9721-43b6-ae31-6b239b60da03" (UID: "b49ecb3d-9721-43b6-ae31-6b239b60da03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:45:23.960888 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:23.960858 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b49ecb3d-9721-43b6-ae31-6b239b60da03-kube-api-access-l6vf7" (OuterVolumeSpecName: "kube-api-access-l6vf7") pod "b49ecb3d-9721-43b6-ae31-6b239b60da03" (UID: "b49ecb3d-9721-43b6-ae31-6b239b60da03"). InnerVolumeSpecName "kube-api-access-l6vf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:45:24.059465 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.059439 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq598\" (UniqueName: \"kubernetes.io/projected/d9114ba2-53e6-487f-a528-0c0c81a9df3f-kube-api-access-xq598\") pod \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " Jul 24 22:45:24.059465 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.059467 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9114ba2-53e6-487f-a528-0c0c81a9df3f-plugins-config-volume\") pod \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\" (UID: \"d9114ba2-53e6-487f-a528-0c0c81a9df3f\") " Jul 24 22:45:24.059671 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.059595 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b49ecb3d-9721-43b6-ae31-6b239b60da03-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:24.059671 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.059605 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6vf7\" (UniqueName: \"kubernetes.io/projected/b49ecb3d-9721-43b6-ae31-6b239b60da03-kube-api-access-l6vf7\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:24.059902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.059878 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9114ba2-53e6-487f-a528-0c0c81a9df3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9114ba2-53e6-487f-a528-0c0c81a9df3f" (UID: "d9114ba2-53e6-487f-a528-0c0c81a9df3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:45:24.061648 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.061627 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9114ba2-53e6-487f-a528-0c0c81a9df3f-kube-api-access-xq598" (OuterVolumeSpecName: "kube-api-access-xq598") pod "d9114ba2-53e6-487f-a528-0c0c81a9df3f" (UID: "d9114ba2-53e6-487f-a528-0c0c81a9df3f"). InnerVolumeSpecName "kube-api-access-xq598". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:45:24.159918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.159890 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq598\" (UniqueName: \"kubernetes.io/projected/d9114ba2-53e6-487f-a528-0c0c81a9df3f-kube-api-access-xq598\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:24.159918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.159913 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9114ba2-53e6-487f-a528-0c0c81a9df3f-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:24.403184 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.403139 2554 generic.go:356] "Generic (PLEG): container finished" podID="d9114ba2-53e6-487f-a528-0c0c81a9df3f" containerID="1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92" exitCode=0 Jul 24 22:45:24.403280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.403204 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" Jul 24 22:45:24.403280 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.403245 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" event={"ID":"d9114ba2-53e6-487f-a528-0c0c81a9df3f","Type":"ContainerDied","Data":"1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92"} Jul 24 22:45:24.403371 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.403292 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc" event={"ID":"d9114ba2-53e6-487f-a528-0c0c81a9df3f","Type":"ContainerDied","Data":"1cda5bda78560648aed104caec6a4468bcf0485981c7075f55653bf86ed804a4"} Jul 24 22:45:24.403371 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.403321 2554 scope.go:117] "RemoveContainer" containerID="1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92" Jul 24 22:45:24.404576 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.404558 2554 generic.go:356] "Generic (PLEG): container finished" podID="b49ecb3d-9721-43b6-ae31-6b239b60da03" containerID="e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f" exitCode=0 Jul 24 22:45:24.404658 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.404607 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" Jul 24 22:45:24.404658 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.404615 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" event={"ID":"b49ecb3d-9721-43b6-ae31-6b239b60da03","Type":"ContainerDied","Data":"e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f"} Jul 24 22:45:24.404658 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.404653 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq" event={"ID":"b49ecb3d-9721-43b6-ae31-6b239b60da03","Type":"ContainerDied","Data":"66b7570b592a71f807eb7e8977fd158c9fa9f47c4ed05e044e534147251e1338"} Jul 24 22:45:24.413572 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.413555 2554 scope.go:117] "RemoveContainer" containerID="1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92" Jul 24 22:45:24.413854 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:45:24.413838 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92\": container with ID starting with 1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92 not found: ID does not exist" containerID="1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92" Jul 24 22:45:24.413923 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.413867 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92"} err="failed to get container status \"1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92\": rpc error: code = NotFound desc = could not find container \"1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92\": container with ID starting with 1909455a9ce3d531cbe914aae66472c8b071be55c91ca337d81ab950e9385e92 not found: ID does not exist" Jul 24 22:45:24.413923 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.413890 2554 scope.go:117] "RemoveContainer" containerID="e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f" Jul 24 22:45:24.421788 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.421770 2554 scope.go:117] "RemoveContainer" containerID="e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f" Jul 24 22:45:24.422024 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:45:24.422008 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f\": container with ID starting with e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f not found: ID does not exist" containerID="e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f" Jul 24 22:45:24.422063 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.422035 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f"} err="failed to get container status \"e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f\": rpc error: code = NotFound desc = could not find container \"e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f\": container with ID starting with e0414ca49de81a0575031e48a236e26bf2be83797d4bfc38848e0507d1993d7f not found: ID does not exist" Jul 24 22:45:24.425614 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.425594 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc"] Jul 24 22:45:24.430612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.430591 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-2db675aa-86f8bb4b7b-7w2cc"] Jul 24 22:45:24.447109 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.447080 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq"] Jul 24 22:45:24.453062 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:24.453041 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-2db675aa-64c548df76-tjrrq"] Jul 24 22:45:25.175041 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:25.175012 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b49ecb3d-9721-43b6-ae31-6b239b60da03" path="/var/lib/kubelet/pods/b49ecb3d-9721-43b6-ae31-6b239b60da03/volumes" Jul 24 22:45:25.175363 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:25.175345 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9114ba2-53e6-487f-a528-0c0c81a9df3f" path="/var/lib/kubelet/pods/d9114ba2-53e6-487f-a528-0c0c81a9df3f/volumes" Jul 24 22:45:29.488621 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:29.488573 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh"] Jul 24 22:45:29.489078 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:29.488840 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" containerID="cri-o://95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78" gracePeriod=30 Jul 24 22:45:30.205725 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.205694 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw"] Jul 24 22:45:30.205972 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.205961 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" Jul 24 22:45:30.206019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.205974 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" Jul 24 22:45:30.206019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.205991 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49ecb3d-9721-43b6-ae31-6b239b60da03" containerName="payload-processing" Jul 24 22:45:30.206019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.205997 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49ecb3d-9721-43b6-ae31-6b239b60da03" containerName="payload-processing" Jul 24 22:45:30.206019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.206009 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9114ba2-53e6-487f-a528-0c0c81a9df3f" containerName="payload-pre-processing" Jul 24 22:45:30.206019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.206014 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9114ba2-53e6-487f-a528-0c0c81a9df3f" containerName="payload-pre-processing" Jul 24 22:45:30.206195 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.206059 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49ecb3d-9721-43b6-ae31-6b239b60da03" containerName="payload-processing" Jul 24 22:45:30.206195 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.206067 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" Jul 24 22:45:30.206195 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.206071 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9114ba2-53e6-487f-a528-0c0c81a9df3f" containerName="payload-pre-processing" Jul 24 22:45:30.206195 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.206078 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" Jul 24 22:45:30.208994 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.208978 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.212118 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.212099 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-869c695e-openshift-default-dockercfg-vss4x\"" Jul 24 22:45:30.232242 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.232220 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw"] Jul 24 22:45:30.303830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.303799 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-certs\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.303830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.303832 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/875b9773-e56e-4301-9e57-7321787e8fa8-istio-podinfo\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.303849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m7rv\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-kube-api-access-2m7rv\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.303875 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-socket\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.303941 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-envoy\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.303982 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-data\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.304078 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-istio-token\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304337 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.304122 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/875b9773-e56e-4301-9e57-7321787e8fa8-istiod-ca-cert\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.304337 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.304171 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-credential-socket\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405248 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-certs\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405279 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/875b9773-e56e-4301-9e57-7321787e8fa8-istio-podinfo\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405477 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405295 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m7rv\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-kube-api-access-2m7rv\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405477 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405468 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-socket\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405556 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405520 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-envoy\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405595 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405569 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-data\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405591 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-istio-token\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405621 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/875b9773-e56e-4301-9e57-7321787e8fa8-istiod-ca-cert\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405648 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-credential-socket\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405891 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405779 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-certs\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.405891 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405796 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-socket\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.406208 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405909 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-data\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.406208 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.405962 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-credential-socket\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.406372 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.406352 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/875b9773-e56e-4301-9e57-7321787e8fa8-istiod-ca-cert\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.408018 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.407980 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-envoy\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.408187 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.408169 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/875b9773-e56e-4301-9e57-7321787e8fa8-istio-podinfo\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.413097 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.413073 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-istio-token\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.413691 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.413673 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m7rv\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-kube-api-access-2m7rv\") pod \"e2e-derive-869c695e-openshift-default-586f56856d-s2wlw\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.519062 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.518978 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:30.652846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.652820 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw"] Jul 24 22:45:30.655311 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:45:30.655277 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875b9773_e56e_4301_9e57_7321787e8fa8.slice/crio-5a0defb4aff62cc7e0560a839dfc28e05133a52d879bf3acb4adc06b260c9ddd WatchSource:0}: Error finding container 5a0defb4aff62cc7e0560a839dfc28e05133a52d879bf3acb4adc06b260c9ddd: Status 404 returned error can't find the container with id 5a0defb4aff62cc7e0560a839dfc28e05133a52d879bf3acb4adc06b260c9ddd Jul 24 22:45:30.657905 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.657870 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:30.658006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.657947 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:30.658006 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:30.657988 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:31.440254 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:31.440222 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" event={"ID":"875b9773-e56e-4301-9e57-7321787e8fa8","Type":"ContainerStarted","Data":"f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832"} Jul 24 22:45:31.440383 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:31.440278 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" event={"ID":"875b9773-e56e-4301-9e57-7321787e8fa8","Type":"ContainerStarted","Data":"5a0defb4aff62cc7e0560a839dfc28e05133a52d879bf3acb4adc06b260c9ddd"} Jul 24 22:45:31.461905 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:31.461858 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" podStartSLOduration=1.4618451590000001 podStartE2EDuration="1.461845159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:45:31.459071661 +0000 UTC m=+1828.845889706" watchObservedRunningTime="2026-07-24 22:45:31.461845159 +0000 UTC m=+1828.848663203" Jul 24 22:45:31.519833 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:31.519814 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:31.524509 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:31.524484 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:32.445213 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:32.445187 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:32.446215 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:32.446193 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:33.542926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.542900 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj"] Jul 24 22:45:33.543292 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.543249 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" Jul 24 22:45:33.543292 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.543263 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f709883-78b7-4600-80d0-75a8fd8ee368" containerName="istio-proxy" Jul 24 22:45:33.545183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.545162 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.548325 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.548307 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-869c695e\"" Jul 24 22:45:33.548405 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.548318 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-869c695e-dockercfg-zr6bh\"" Jul 24 22:45:33.562439 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.562415 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj"] Jul 24 22:45:33.569331 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.569310 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf"] Jul 24 22:45:33.571323 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.571310 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.586941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.586919 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf"] Jul 24 22:45:33.632159 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.632121 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tbqs\" (UniqueName: \"kubernetes.io/projected/b48f0b81-550e-4cb3-9f0e-c020b4844493-kube-api-access-6tbqs\") pod \"payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.632276 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.632187 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngcfs\" (UniqueName: \"kubernetes.io/projected/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-kube-api-access-ngcfs\") pod \"payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.632349 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.632275 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-plugins-config-volume\") pod \"payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.632404 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.632378 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b48f0b81-550e-4cb3-9f0e-c020b4844493-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.732839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.732803 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-plugins-config-volume\") pod \"payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.733000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.732875 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b48f0b81-550e-4cb3-9f0e-c020b4844493-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.733000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.732914 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tbqs\" (UniqueName: \"kubernetes.io/projected/b48f0b81-550e-4cb3-9f0e-c020b4844493-kube-api-access-6tbqs\") pod \"payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.733000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.732944 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngcfs\" (UniqueName: \"kubernetes.io/projected/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-kube-api-access-ngcfs\") pod \"payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.733581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.733561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b48f0b81-550e-4cb3-9f0e-c020b4844493-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.733651 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.733583 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-plugins-config-volume\") pod \"payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.741651 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.741623 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngcfs\" (UniqueName: \"kubernetes.io/projected/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-kube-api-access-ngcfs\") pod \"payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.741830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.741809 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tbqs\" (UniqueName: \"kubernetes.io/projected/b48f0b81-550e-4cb3-9f0e-c020b4844493-kube-api-access-6tbqs\") pod \"payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.855830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.855771 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:33.881113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.881092 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:33.999505 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:33.999467 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj"] Jul 24 22:45:34.001212 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:45:34.001186 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb48f0b81_550e_4cb3_9f0e_c020b4844493.slice/crio-3af21e135ac85f4d7dcb1a083818a60c2c6a46c355e709fd054f32fd094248e3 WatchSource:0}: Error finding container 3af21e135ac85f4d7dcb1a083818a60c2c6a46c355e709fd054f32fd094248e3: Status 404 returned error can't find the container with id 3af21e135ac85f4d7dcb1a083818a60c2c6a46c355e709fd054f32fd094248e3 Jul 24 22:45:34.034332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.034309 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf"] Jul 24 22:45:34.035628 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:45:34.035600 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b08ff2_80df_49f3_aeb3_97325a32b6b4.slice/crio-5ae80cfceeb101a36d89a274cf193604665118294ab1046c67473098df11e494 WatchSource:0}: Error finding container 5ae80cfceeb101a36d89a274cf193604665118294ab1046c67473098df11e494: Status 404 returned error can't find the container with id 5ae80cfceeb101a36d89a274cf193604665118294ab1046c67473098df11e494 Jul 24 22:45:34.457187 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.457134 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" event={"ID":"b2b08ff2-80df-49f3-aeb3-97325a32b6b4","Type":"ContainerStarted","Data":"c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b"} Jul 24 22:45:34.457377 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.457198 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" event={"ID":"b2b08ff2-80df-49f3-aeb3-97325a32b6b4","Type":"ContainerStarted","Data":"5ae80cfceeb101a36d89a274cf193604665118294ab1046c67473098df11e494"} Jul 24 22:45:34.457377 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.457242 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:34.458869 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.458846 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" event={"ID":"b48f0b81-550e-4cb3-9f0e-c020b4844493","Type":"ContainerStarted","Data":"7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db"} Jul 24 22:45:34.458974 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.458888 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:34.458974 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.458904 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" event={"ID":"b48f0b81-550e-4cb3-9f0e-c020b4844493","Type":"ContainerStarted","Data":"3af21e135ac85f4d7dcb1a083818a60c2c6a46c355e709fd054f32fd094248e3"} Jul 24 22:45:34.472985 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.472937 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" podStartSLOduration=1.472924768 podStartE2EDuration="1.472924768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:45:34.471923764 +0000 UTC m=+1831.858741811" watchObservedRunningTime="2026-07-24 22:45:34.472924768 +0000 UTC m=+1831.859742871" Jul 24 22:45:34.486289 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.486238 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" podStartSLOduration=1.486227242 podStartE2EDuration="1.486227242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:45:34.485921179 +0000 UTC m=+1831.872739224" watchObservedRunningTime="2026-07-24 22:45:34.486227242 +0000 UTC m=+1831.873045299" Jul 24 22:45:34.722485 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.722458 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:45:34.841578 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841549 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-data\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841599 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db6jc\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-kube-api-access-db6jc\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841617 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5281da-5fae-4581-95c2-251c35c766ed-istiod-ca-cert\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841655 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-istio-token\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841678 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-credential-socket\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841888 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841848 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-envoy\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841888 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5281da-5fae-4581-95c2-251c35c766ed-istio-podinfo\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.841997 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841932 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:45:34.841997 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.841947 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-certs\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.842100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842034 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5281da-5fae-4581-95c2-251c35c766ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:45:34.842100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842074 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-socket\") pod \"8c5281da-5fae-4581-95c2-251c35c766ed\" (UID: \"8c5281da-5fae-4581-95c2-251c35c766ed\") " Jul 24 22:45:34.842287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842233 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:45:34.842425 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842402 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:45:34.842506 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842425 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.842506 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842456 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5281da-5fae-4581-95c2-251c35c766ed-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.842506 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842469 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.842765 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.842742 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:45:34.844066 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.844037 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:45:34.844212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.844108 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c5281da-5fae-4581-95c2-251c35c766ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:45:34.844442 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.844422 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-kube-api-access-db6jc" (OuterVolumeSpecName: "kube-api-access-db6jc") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "kube-api-access-db6jc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:45:34.844495 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.844479 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c5281da-5fae-4581-95c2-251c35c766ed" (UID: "8c5281da-5fae-4581-95c2-251c35c766ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:45:34.943780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.943754 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.943780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.943777 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.943910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.943787 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db6jc\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-kube-api-access-db6jc\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.943910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.943796 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5281da-5fae-4581-95c2-251c35c766ed-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.943910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.943804 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5281da-5fae-4581-95c2-251c35c766ed-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:34.943910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:34.943811 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5281da-5fae-4581-95c2-251c35c766ed-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:45:35.467970 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.467873 2554 generic.go:356] "Generic (PLEG): container finished" podID="8c5281da-5fae-4581-95c2-251c35c766ed" containerID="95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78" exitCode=0 Jul 24 22:45:35.467970 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.467930 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" Jul 24 22:45:35.468221 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.467978 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" event={"ID":"8c5281da-5fae-4581-95c2-251c35c766ed","Type":"ContainerDied","Data":"95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78"} Jul 24 22:45:35.468221 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.468029 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh" event={"ID":"8c5281da-5fae-4581-95c2-251c35c766ed","Type":"ContainerDied","Data":"1f918858ab7de394a92a41b8dfe39ad982d7ae9c3dbbf935271a9233bf99f8da"} Jul 24 22:45:35.468221 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.468059 2554 scope.go:117] "RemoveContainer" containerID="95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78" Jul 24 22:45:35.478284 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.478266 2554 scope.go:117] "RemoveContainer" containerID="78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06" Jul 24 22:45:35.490939 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.490677 2554 scope.go:117] "RemoveContainer" containerID="95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78" Jul 24 22:45:35.495495 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.495447 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh"] Jul 24 22:45:35.497557 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:45:35.497532 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78\": container with ID starting with 95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78 not found: ID does not exist" containerID="95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78" Jul 24 22:45:35.497664 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.497564 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78"} err="failed to get container status \"95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78\": rpc error: code = NotFound desc = could not find container \"95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78\": container with ID starting with 95b725085c8e4bec6f0e5b3bceb6499d16750ede7bb42f8a6f1deebe7e201d78 not found: ID does not exist" Jul 24 22:45:35.497664 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.497586 2554 scope.go:117] "RemoveContainer" containerID="78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06" Jul 24 22:45:35.497842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.497830 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2db675aa-openshift-default-7d4674f988-rsdgh"] Jul 24 22:45:35.498031 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:45:35.498004 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06\": container with ID starting with 78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06 not found: ID does not exist" containerID="78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06" Jul 24 22:45:35.498143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:35.498037 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06"} err="failed to get container status \"78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06\": rpc error: code = NotFound desc = could not find container \"78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06\": container with ID starting with 78b4106be7123f682f3767b993c24054f12a3032a6e50febc65b973aac3a3f06 not found: ID does not exist" Jul 24 22:45:37.176530 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:37.176498 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" path="/var/lib/kubelet/pods/8c5281da-5fae-4581-95c2-251c35c766ed/volumes" Jul 24 22:45:45.470300 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:45.470268 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:45:45.470754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:45.470339 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:45:47.447092 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.447064 2554 patch_prober.go:28] interesting pod/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 22:45:47.447562 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.447128 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 22:45:47.530929 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.530897 2554 generic.go:356] "Generic (PLEG): container finished" podID="875b9773-e56e-4301-9e57-7321787e8fa8" containerID="f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832" exitCode=0 Jul 24 22:45:47.531056 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.530970 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" event={"ID":"875b9773-e56e-4301-9e57-7321787e8fa8","Type":"ContainerDied","Data":"f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832"} Jul 24 22:45:47.531375 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.531361 2554 scope.go:117] "RemoveContainer" containerID="f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832" Jul 24 22:45:47.532394 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.532366 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:47.532481 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.532449 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:47.532518 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:47.532493 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:45:48.538027 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:48.537991 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" event={"ID":"875b9773-e56e-4301-9e57-7321787e8fa8","Type":"ContainerStarted","Data":"f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea"} Jul 24 22:45:49.519660 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:49.519629 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:49.519833 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:49.519759 2554 patch_prober.go:28] interesting pod/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 22:45:49.519833 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:49.519809 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 22:45:50.519831 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:50.519794 2554 patch_prober.go:28] interesting pod/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 22:45:50.520267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:50.519859 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 22:45:51.519579 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:51.519543 2554 patch_prober.go:28] interesting pod/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 24 22:45:51.519764 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:51.519597 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 24 22:45:52.523647 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:52.523614 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:52.524022 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:52.523962 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:45:52.524757 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:45:52.524739 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:46:08.095569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.095533 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf"] Jul 24 22:46:08.095946 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.095818 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" podUID="b2b08ff2-80df-49f3-aeb3-97325a32b6b4" containerName="payload-processing" containerID="cri-o://c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b" gracePeriod=30 Jul 24 22:46:08.101106 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.101082 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj"] Jul 24 22:46:08.101371 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.101308 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" podUID="b48f0b81-550e-4cb3-9f0e-c020b4844493" containerName="payload-pre-processing" containerID="cri-o://7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db" gracePeriod=30 Jul 24 22:46:08.371760 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.371732 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:46:08.403370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.403350 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:46:08.516666 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.516637 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngcfs\" (UniqueName: \"kubernetes.io/projected/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-kube-api-access-ngcfs\") pod \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " Jul 24 22:46:08.516822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.516711 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tbqs\" (UniqueName: \"kubernetes.io/projected/b48f0b81-550e-4cb3-9f0e-c020b4844493-kube-api-access-6tbqs\") pod \"b48f0b81-550e-4cb3-9f0e-c020b4844493\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " Jul 24 22:46:08.516822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.516738 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-plugins-config-volume\") pod \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\" (UID: \"b2b08ff2-80df-49f3-aeb3-97325a32b6b4\") " Jul 24 22:46:08.516822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.516754 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b48f0b81-550e-4cb3-9f0e-c020b4844493-plugins-config-volume\") pod \"b48f0b81-550e-4cb3-9f0e-c020b4844493\" (UID: \"b48f0b81-550e-4cb3-9f0e-c020b4844493\") " Jul 24 22:46:08.517112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.517085 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2b08ff2-80df-49f3-aeb3-97325a32b6b4" (UID: "b2b08ff2-80df-49f3-aeb3-97325a32b6b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:46:08.517240 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.517136 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b48f0b81-550e-4cb3-9f0e-c020b4844493-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b48f0b81-550e-4cb3-9f0e-c020b4844493" (UID: "b48f0b81-550e-4cb3-9f0e-c020b4844493"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:46:08.518914 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.518889 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b48f0b81-550e-4cb3-9f0e-c020b4844493-kube-api-access-6tbqs" (OuterVolumeSpecName: "kube-api-access-6tbqs") pod "b48f0b81-550e-4cb3-9f0e-c020b4844493" (UID: "b48f0b81-550e-4cb3-9f0e-c020b4844493"). InnerVolumeSpecName "kube-api-access-6tbqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:46:08.518914 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.518903 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-kube-api-access-ngcfs" (OuterVolumeSpecName: "kube-api-access-ngcfs") pod "b2b08ff2-80df-49f3-aeb3-97325a32b6b4" (UID: "b2b08ff2-80df-49f3-aeb3-97325a32b6b4"). InnerVolumeSpecName "kube-api-access-ngcfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:46:08.617561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.617496 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tbqs\" (UniqueName: \"kubernetes.io/projected/b48f0b81-550e-4cb3-9f0e-c020b4844493-kube-api-access-6tbqs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:08.617561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.617522 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:08.617561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.617531 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b48f0b81-550e-4cb3-9f0e-c020b4844493-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:08.617561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.617539 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngcfs\" (UniqueName: \"kubernetes.io/projected/b2b08ff2-80df-49f3-aeb3-97325a32b6b4-kube-api-access-ngcfs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:08.638769 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.638744 2554 generic.go:356] "Generic (PLEG): container finished" podID="b2b08ff2-80df-49f3-aeb3-97325a32b6b4" containerID="c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b" exitCode=0 Jul 24 22:46:08.638895 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.638783 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" Jul 24 22:46:08.638895 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.638825 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" event={"ID":"b2b08ff2-80df-49f3-aeb3-97325a32b6b4","Type":"ContainerDied","Data":"c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b"} Jul 24 22:46:08.638895 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.638877 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf" event={"ID":"b2b08ff2-80df-49f3-aeb3-97325a32b6b4","Type":"ContainerDied","Data":"5ae80cfceeb101a36d89a274cf193604665118294ab1046c67473098df11e494"} Jul 24 22:46:08.639029 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.638904 2554 scope.go:117] "RemoveContainer" containerID="c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b" Jul 24 22:46:08.640208 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.640186 2554 generic.go:356] "Generic (PLEG): container finished" podID="b48f0b81-550e-4cb3-9f0e-c020b4844493" containerID="7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db" exitCode=0 Jul 24 22:46:08.640298 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.640221 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" event={"ID":"b48f0b81-550e-4cb3-9f0e-c020b4844493","Type":"ContainerDied","Data":"7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db"} Jul 24 22:46:08.640298 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.640258 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" event={"ID":"b48f0b81-550e-4cb3-9f0e-c020b4844493","Type":"ContainerDied","Data":"3af21e135ac85f4d7dcb1a083818a60c2c6a46c355e709fd054f32fd094248e3"} Jul 24 22:46:08.640298 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.640226 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj" Jul 24 22:46:08.648938 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.648913 2554 scope.go:117] "RemoveContainer" containerID="c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b" Jul 24 22:46:08.649270 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:46:08.649250 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b\": container with ID starting with c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b not found: ID does not exist" containerID="c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b" Jul 24 22:46:08.649346 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.649276 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b"} err="failed to get container status \"c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b\": rpc error: code = NotFound desc = could not find container \"c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b\": container with ID starting with c6fa14a9708f0e3c04773e10419037d732146596f2badefa9ae48d8f10e1ab3b not found: ID does not exist" Jul 24 22:46:08.649346 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.649291 2554 scope.go:117] "RemoveContainer" containerID="7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db" Jul 24 22:46:08.657047 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.657033 2554 scope.go:117] "RemoveContainer" containerID="7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db" Jul 24 22:46:08.657299 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:46:08.657280 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db\": container with ID starting with 7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db not found: ID does not exist" containerID="7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db" Jul 24 22:46:08.657355 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.657306 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db"} err="failed to get container status \"7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db\": rpc error: code = NotFound desc = could not find container \"7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db\": container with ID starting with 7f6bbbe5a45a10a0f25dec7364002a509127c49d016f135ee9eb192e943ae5db not found: ID does not exist" Jul 24 22:46:08.663270 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.663248 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj"] Jul 24 22:46:08.668839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.668821 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-869c695e-5f8664f744-r7tcj"] Jul 24 22:46:08.678743 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.678725 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf"] Jul 24 22:46:08.684287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:08.684268 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-869c695e-5bc7594d58-j7bmf"] Jul 24 22:46:09.175137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:09.175107 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b08ff2-80df-49f3-aeb3-97325a32b6b4" path="/var/lib/kubelet/pods/b2b08ff2-80df-49f3-aeb3-97325a32b6b4/volumes" Jul 24 22:46:09.175474 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:09.175461 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b48f0b81-550e-4cb3-9f0e-c020b4844493" path="/var/lib/kubelet/pods/b48f0b81-550e-4cb3-9f0e-c020b4844493/volumes" Jul 24 22:46:12.837471 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:12.837440 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw"] Jul 24 22:46:12.837918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:12.837655 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" containerID="cri-o://f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea" gracePeriod=30 Jul 24 22:46:18.086921 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.086899 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:46:18.186379 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186354 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/875b9773-e56e-4301-9e57-7321787e8fa8-istiod-ca-cert\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186577 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186392 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m7rv\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-kube-api-access-2m7rv\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186577 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186418 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-data\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186577 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186543 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-envoy\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186590 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-certs\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186626 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-credential-socket\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186664 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-socket\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186702 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-istio-token\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186759 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186729 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/875b9773-e56e-4301-9e57-7321787e8fa8-istio-podinfo\") pod \"875b9773-e56e-4301-9e57-7321787e8fa8\" (UID: \"875b9773-e56e-4301-9e57-7321787e8fa8\") " Jul 24 22:46:18.186995 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186826 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/875b9773-e56e-4301-9e57-7321787e8fa8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:46:18.186995 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186901 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:46:18.186995 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186916 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:46:18.186995 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.186931 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:46:18.187185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.187001 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.187185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.187021 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.187185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.187035 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.187185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.187078 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/875b9773-e56e-4301-9e57-7321787e8fa8-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.187511 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.187478 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-data" (OuterVolumeSpecName: "istio-data") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:46:18.188748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.188724 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-kube-api-access-2m7rv" (OuterVolumeSpecName: "kube-api-access-2m7rv") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "kube-api-access-2m7rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:46:18.188956 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.188936 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-istio-token" (OuterVolumeSpecName: "istio-token") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:46:18.189304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.189278 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/875b9773-e56e-4301-9e57-7321787e8fa8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:46:18.189304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.189286 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "875b9773-e56e-4301-9e57-7321787e8fa8" (UID: "875b9773-e56e-4301-9e57-7321787e8fa8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:46:18.293352 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.288977 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.293352 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.289009 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/875b9773-e56e-4301-9e57-7321787e8fa8-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.293352 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.289023 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m7rv\" (UniqueName: \"kubernetes.io/projected/875b9773-e56e-4301-9e57-7321787e8fa8-kube-api-access-2m7rv\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.293352 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.289043 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.293352 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.289056 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/875b9773-e56e-4301-9e57-7321787e8fa8-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:46:18.690297 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.690263 2554 generic.go:356] "Generic (PLEG): container finished" podID="875b9773-e56e-4301-9e57-7321787e8fa8" containerID="f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea" exitCode=0 Jul 24 22:46:18.690451 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.690314 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" event={"ID":"875b9773-e56e-4301-9e57-7321787e8fa8","Type":"ContainerDied","Data":"f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea"} Jul 24 22:46:18.690451 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.690324 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" Jul 24 22:46:18.690451 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.690380 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw" event={"ID":"875b9773-e56e-4301-9e57-7321787e8fa8","Type":"ContainerDied","Data":"5a0defb4aff62cc7e0560a839dfc28e05133a52d879bf3acb4adc06b260c9ddd"} Jul 24 22:46:18.690451 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.690401 2554 scope.go:117] "RemoveContainer" containerID="f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea" Jul 24 22:46:18.699753 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.699739 2554 scope.go:117] "RemoveContainer" containerID="f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832" Jul 24 22:46:18.707668 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.707649 2554 scope.go:117] "RemoveContainer" containerID="f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea" Jul 24 22:46:18.707895 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:46:18.707877 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea\": container with ID starting with f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea not found: ID does not exist" containerID="f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea" Jul 24 22:46:18.707943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.707903 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea"} err="failed to get container status \"f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea\": rpc error: code = NotFound desc = could not find container \"f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea\": container with ID starting with f3f6384fb478115600cdcc87c31b2f7d5646ec3441d5abe4a1b7f6703a667cea not found: ID does not exist" Jul 24 22:46:18.707943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.707921 2554 scope.go:117] "RemoveContainer" containerID="f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832" Jul 24 22:46:18.708200 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:46:18.708176 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832\": container with ID starting with f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832 not found: ID does not exist" containerID="f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832" Jul 24 22:46:18.708292 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.708207 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832"} err="failed to get container status \"f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832\": rpc error: code = NotFound desc = could not find container \"f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832\": container with ID starting with f279f2369c78ef722314839c46ebc3a30ca05cc9369305ac12a88bdc3d6aa832 not found: ID does not exist" Jul 24 22:46:18.712163 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.712124 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw"] Jul 24 22:46:18.717505 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:18.717484 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-869c695e-openshift-default-586f56856d-s2wlw"] Jul 24 22:46:19.176078 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:46:19.176046 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" path="/var/lib/kubelet/pods/875b9773-e56e-4301-9e57-7321787e8fa8/volumes" Jul 24 22:47:48.949542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949507 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv"] Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949811 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949823 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949833 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949839 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949847 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b48f0b81-550e-4cb3-9f0e-c020b4844493" containerName="payload-pre-processing" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949857 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b48f0b81-550e-4cb3-9f0e-c020b4844493" containerName="payload-pre-processing" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949866 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949872 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949878 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b08ff2-80df-49f3-aeb3-97325a32b6b4" containerName="payload-processing" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949883 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b08ff2-80df-49f3-aeb3-97325a32b6b4" containerName="payload-processing" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949943 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949952 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b48f0b81-550e-4cb3-9f0e-c020b4844493" containerName="payload-pre-processing" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949960 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949966 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b08ff2-80df-49f3-aeb3-97325a32b6b4" containerName="payload-processing" Jul 24 22:47:48.950021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.949971 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c5281da-5fae-4581-95c2-251c35c766ed" containerName="istio-proxy" Jul 24 22:47:48.952837 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.952814 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:48.955577 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.955558 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-85e566ec-openshift-default-dockercfg-zwqnh\"" Jul 24 22:47:48.974538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:48.974507 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv"] Jul 24 22:47:49.071771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.071735 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-certs\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.071960 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.071778 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fktt9\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-kube-api-access-fktt9\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.071960 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.071802 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-envoy\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.071960 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.071905 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-credential-socket\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.071960 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.071954 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-token\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.072175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.071997 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istiod-ca-cert\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.072175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.072033 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-socket\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.072175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.072064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-data\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.072175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.072109 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-podinfo\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.172758 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.172727 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-token\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.172758 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.172761 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istiod-ca-cert\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.172780 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-socket\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.172795 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-data\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.172820 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-podinfo\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.172842 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-certs\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173107 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fktt9\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-kube-api-access-fktt9\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173174 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-envoy\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173232 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-credential-socket\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173421 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173307 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-data\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173593 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173569 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-certs\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173593 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173580 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istiod-ca-cert\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173593 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173585 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-socket\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.173771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.173690 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-credential-socket\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.175624 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.175605 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-podinfo\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.175976 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.175961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-envoy\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.180963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.180941 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-token\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.182121 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.182097 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fktt9\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-kube-api-access-fktt9\") pod \"e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.263021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.262929 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:49.403401 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.403378 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv"] Jul 24 22:47:49.405488 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:47:49.405458 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbf7f6cc_7ab1_4e88_9aa8_24a61c7e9209.slice/crio-4d851be935dda9866d761ea4583038345d28674180d65461b8ae57582d4009b9 WatchSource:0}: Error finding container 4d851be935dda9866d761ea4583038345d28674180d65461b8ae57582d4009b9: Status 404 returned error can't find the container with id 4d851be935dda9866d761ea4583038345d28674180d65461b8ae57582d4009b9 Jul 24 22:47:49.408068 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.408028 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:47:49.408224 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.408112 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:47:49.408224 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:49.408176 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:47:50.112733 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:50.112696 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" event={"ID":"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209","Type":"ContainerStarted","Data":"17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a"} Jul 24 22:47:50.113121 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:50.112748 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" event={"ID":"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209","Type":"ContainerStarted","Data":"4d851be935dda9866d761ea4583038345d28674180d65461b8ae57582d4009b9"} Jul 24 22:47:50.149530 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:50.149484 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podStartSLOduration=2.149469784 podStartE2EDuration="2.149469784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:47:50.14688696 +0000 UTC m=+1967.533705008" watchObservedRunningTime="2026-07-24 22:47:50.149469784 +0000 UTC m=+1967.536287830" Jul 24 22:47:50.263751 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:50.263725 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:50.272789 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:50.272763 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:51.117405 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:51.117379 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:51.118281 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:51.118259 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:47:52.859000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.858971 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7"] Jul 24 22:47:52.859377 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.859268 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" Jul 24 22:47:52.859377 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.859280 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" Jul 24 22:47:52.859377 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.859345 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="875b9773-e56e-4301-9e57-7321787e8fa8" containerName="istio-proxy" Jul 24 22:47:52.862303 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.862287 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:52.865067 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.865045 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-85e566ec\"" Jul 24 22:47:52.865208 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.865053 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-85e566ec-dockercfg-sfhzh\"" Jul 24 22:47:52.872023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.872000 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7"] Jul 24 22:47:52.894320 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.894293 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt"] Jul 24 22:47:52.897338 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.897320 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:52.905902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:52.905879 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt"] Jul 24 22:47:53.005681 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.005639 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwsks\" (UniqueName: \"kubernetes.io/projected/4eac7522-f345-4051-ba91-e0b0c5ec07cf-kube-api-access-pwsks\") pod \"payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.005859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.005756 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac7522-f345-4051-ba91-e0b0c5ec07cf-plugins-config-volume\") pod \"payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.005859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.005803 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsdl4\" (UniqueName: \"kubernetes.io/projected/ac7f4bda-acd3-4768-a559-64008e6630ee-kube-api-access-lsdl4\") pod \"payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.005859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.005831 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7f4bda-acd3-4768-a559-64008e6630ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.107032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.106952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac7522-f345-4051-ba91-e0b0c5ec07cf-plugins-config-volume\") pod \"payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.107032 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.107008 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsdl4\" (UniqueName: \"kubernetes.io/projected/ac7f4bda-acd3-4768-a559-64008e6630ee-kube-api-access-lsdl4\") pod \"payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.107260 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.107043 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7f4bda-acd3-4768-a559-64008e6630ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.107260 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.107081 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwsks\" (UniqueName: \"kubernetes.io/projected/4eac7522-f345-4051-ba91-e0b0c5ec07cf-kube-api-access-pwsks\") pod \"payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.107707 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.107687 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac7522-f345-4051-ba91-e0b0c5ec07cf-plugins-config-volume\") pod \"payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.107829 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.107813 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7f4bda-acd3-4768-a559-64008e6630ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.115396 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.115340 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwsks\" (UniqueName: \"kubernetes.io/projected/4eac7522-f345-4051-ba91-e0b0c5ec07cf-kube-api-access-pwsks\") pod \"payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.115497 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.115486 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsdl4\" (UniqueName: \"kubernetes.io/projected/ac7f4bda-acd3-4768-a559-64008e6630ee-kube-api-access-lsdl4\") pod \"payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.173404 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.173385 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:53.207893 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.207862 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:53.325639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.325109 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7"] Jul 24 22:47:53.327769 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:47:53.327742 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7f4bda_acd3_4768_a559_64008e6630ee.slice/crio-03f89bd951ebae0c7a8cbeb3faff566a2d313fd88844de36e982f3ac6849a646 WatchSource:0}: Error finding container 03f89bd951ebae0c7a8cbeb3faff566a2d313fd88844de36e982f3ac6849a646: Status 404 returned error can't find the container with id 03f89bd951ebae0c7a8cbeb3faff566a2d313fd88844de36e982f3ac6849a646 Jul 24 22:47:53.352377 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:53.352355 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt"] Jul 24 22:47:53.354210 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:47:53.354186 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eac7522_f345_4051_ba91_e0b0c5ec07cf.slice/crio-ce7b0f387185986f275101b94f096a5d352e3dc2f97f7f2ccbd85ab335fe4bee WatchSource:0}: Error finding container ce7b0f387185986f275101b94f096a5d352e3dc2f97f7f2ccbd85ab335fe4bee: Status 404 returned error can't find the container with id ce7b0f387185986f275101b94f096a5d352e3dc2f97f7f2ccbd85ab335fe4bee Jul 24 22:47:54.134313 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.134278 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" event={"ID":"4eac7522-f345-4051-ba91-e0b0c5ec07cf","Type":"ContainerStarted","Data":"2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95"} Jul 24 22:47:54.134942 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.134334 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" event={"ID":"4eac7522-f345-4051-ba91-e0b0c5ec07cf","Type":"ContainerStarted","Data":"ce7b0f387185986f275101b94f096a5d352e3dc2f97f7f2ccbd85ab335fe4bee"} Jul 24 22:47:54.134942 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.134429 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:47:54.135851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.135830 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" event={"ID":"ac7f4bda-acd3-4768-a559-64008e6630ee","Type":"ContainerStarted","Data":"3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb"} Jul 24 22:47:54.135966 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.135869 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" event={"ID":"ac7f4bda-acd3-4768-a559-64008e6630ee","Type":"ContainerStarted","Data":"03f89bd951ebae0c7a8cbeb3faff566a2d313fd88844de36e982f3ac6849a646"} Jul 24 22:47:54.135966 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.135933 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:47:54.155505 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.155456 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" podStartSLOduration=2.155437656 podStartE2EDuration="2.155437656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:47:54.153350231 +0000 UTC m=+1971.540168277" watchObservedRunningTime="2026-07-24 22:47:54.155437656 +0000 UTC m=+1971.542255707" Jul 24 22:47:54.176821 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:47:54.176772 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" podStartSLOduration=2.176754768 podStartE2EDuration="2.176754768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:47:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:47:54.173638727 +0000 UTC m=+1971.560456774" watchObservedRunningTime="2026-07-24 22:47:54.176754768 +0000 UTC m=+1971.563572818" Jul 24 22:48:05.142201 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.142133 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:48:05.142581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.142355 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:48:05.201597 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.201567 2554 generic.go:356] "Generic (PLEG): container finished" podID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerID="17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a" exitCode=0 Jul 24 22:48:05.201597 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.201602 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" event={"ID":"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209","Type":"ContainerDied","Data":"17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a"} Jul 24 22:48:05.201976 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.201959 2554 scope.go:117] "RemoveContainer" containerID="17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a" Jul 24 22:48:05.203003 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.202976 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:48:05.203114 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.203053 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:48:05.203114 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:05.203099 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:48:06.117842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:06.117802 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:06.208089 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:06.208054 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" event={"ID":"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209","Type":"ContainerStarted","Data":"09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e"} Jul 24 22:48:06.264115 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:06.264068 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:06.265507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:06.265475 2554 patch_prober.go:28] interesting pod/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 22:48:06.265634 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:06.265543 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 22:48:07.264105 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:07.264057 2554 patch_prober.go:28] interesting pod/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 22:48:07.264715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:07.264666 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 22:48:08.263911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:08.263866 2554 patch_prober.go:28] interesting pod/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 22:48:08.264125 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:08.263942 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 22:48:09.263533 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:09.263494 2554 patch_prober.go:28] interesting pod/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 24 22:48:09.263746 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:09.263571 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 24 22:48:10.268008 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:10.267970 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:10.268399 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:10.268278 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:10.268882 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:10.268865 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:13.834553 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:13.834516 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt"] Jul 24 22:48:13.835016 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:13.834752 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" podUID="4eac7522-f345-4051-ba91-e0b0c5ec07cf" containerName="payload-processing" containerID="cri-o://2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95" gracePeriod=30 Jul 24 22:48:13.839810 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:13.839781 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7"] Jul 24 22:48:13.840291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:13.840240 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" podUID="ac7f4bda-acd3-4768-a559-64008e6630ee" containerName="payload-pre-processing" containerID="cri-o://3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb" gracePeriod=30 Jul 24 22:48:14.109661 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.109641 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:48:14.112260 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.112245 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:48:14.172611 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.172584 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac7522-f345-4051-ba91-e0b0c5ec07cf-plugins-config-volume\") pod \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " Jul 24 22:48:14.172740 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.172649 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7f4bda-acd3-4768-a559-64008e6630ee-plugins-config-volume\") pod \"ac7f4bda-acd3-4768-a559-64008e6630ee\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " Jul 24 22:48:14.172740 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.172715 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwsks\" (UniqueName: \"kubernetes.io/projected/4eac7522-f345-4051-ba91-e0b0c5ec07cf-kube-api-access-pwsks\") pod \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\" (UID: \"4eac7522-f345-4051-ba91-e0b0c5ec07cf\") " Jul 24 22:48:14.172835 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.172745 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsdl4\" (UniqueName: \"kubernetes.io/projected/ac7f4bda-acd3-4768-a559-64008e6630ee-kube-api-access-lsdl4\") pod \"ac7f4bda-acd3-4768-a559-64008e6630ee\" (UID: \"ac7f4bda-acd3-4768-a559-64008e6630ee\") " Jul 24 22:48:14.173009 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.172989 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eac7522-f345-4051-ba91-e0b0c5ec07cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4eac7522-f345-4051-ba91-e0b0c5ec07cf" (UID: "4eac7522-f345-4051-ba91-e0b0c5ec07cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:48:14.173083 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.173055 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7f4bda-acd3-4768-a559-64008e6630ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac7f4bda-acd3-4768-a559-64008e6630ee" (UID: "ac7f4bda-acd3-4768-a559-64008e6630ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:48:14.174906 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.174879 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eac7522-f345-4051-ba91-e0b0c5ec07cf-kube-api-access-pwsks" (OuterVolumeSpecName: "kube-api-access-pwsks") pod "4eac7522-f345-4051-ba91-e0b0c5ec07cf" (UID: "4eac7522-f345-4051-ba91-e0b0c5ec07cf"). InnerVolumeSpecName "kube-api-access-pwsks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:48:14.175000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.174943 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7f4bda-acd3-4768-a559-64008e6630ee-kube-api-access-lsdl4" (OuterVolumeSpecName: "kube-api-access-lsdl4") pod "ac7f4bda-acd3-4768-a559-64008e6630ee" (UID: "ac7f4bda-acd3-4768-a559-64008e6630ee"). InnerVolumeSpecName "kube-api-access-lsdl4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:48:14.244473 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.244445 2554 generic.go:356] "Generic (PLEG): container finished" podID="ac7f4bda-acd3-4768-a559-64008e6630ee" containerID="3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb" exitCode=0 Jul 24 22:48:14.244592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.244493 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" Jul 24 22:48:14.244592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.244533 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" event={"ID":"ac7f4bda-acd3-4768-a559-64008e6630ee","Type":"ContainerDied","Data":"3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb"} Jul 24 22:48:14.244592 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.244586 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7" event={"ID":"ac7f4bda-acd3-4768-a559-64008e6630ee","Type":"ContainerDied","Data":"03f89bd951ebae0c7a8cbeb3faff566a2d313fd88844de36e982f3ac6849a646"} Jul 24 22:48:14.244747 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.244618 2554 scope.go:117] "RemoveContainer" containerID="3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb" Jul 24 22:48:14.246296 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.246269 2554 generic.go:356] "Generic (PLEG): container finished" podID="4eac7522-f345-4051-ba91-e0b0c5ec07cf" containerID="2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95" exitCode=0 Jul 24 22:48:14.246411 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.246300 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" Jul 24 22:48:14.246411 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.246318 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" event={"ID":"4eac7522-f345-4051-ba91-e0b0c5ec07cf","Type":"ContainerDied","Data":"2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95"} Jul 24 22:48:14.246411 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.246371 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt" event={"ID":"4eac7522-f345-4051-ba91-e0b0c5ec07cf","Type":"ContainerDied","Data":"ce7b0f387185986f275101b94f096a5d352e3dc2f97f7f2ccbd85ab335fe4bee"} Jul 24 22:48:14.254626 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.254605 2554 scope.go:117] "RemoveContainer" containerID="3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb" Jul 24 22:48:14.254893 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:48:14.254873 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb\": container with ID starting with 3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb not found: ID does not exist" containerID="3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb" Jul 24 22:48:14.254948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.254901 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb"} err="failed to get container status \"3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb\": rpc error: code = NotFound desc = could not find container \"3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb\": container with ID starting with 3a2567838164d7dcb59ffa46f5da865004e61be582cb73ced1e597dadc0aacfb not found: ID does not exist" Jul 24 22:48:14.254948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.254917 2554 scope.go:117] "RemoveContainer" containerID="2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95" Jul 24 22:48:14.262513 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.262496 2554 scope.go:117] "RemoveContainer" containerID="2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95" Jul 24 22:48:14.262759 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:48:14.262742 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95\": container with ID starting with 2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95 not found: ID does not exist" containerID="2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95" Jul 24 22:48:14.262820 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.262771 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95"} err="failed to get container status \"2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95\": rpc error: code = NotFound desc = could not find container \"2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95\": container with ID starting with 2a584a1cdfd626ce31cca8a393aa0e92c99b537d8b9657bb7be6153c1cdcce95 not found: ID does not exist" Jul 24 22:48:14.267260 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.267236 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7"] Jul 24 22:48:14.270792 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.270771 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85e566ec-6f855fb8d-q4vq7"] Jul 24 22:48:14.273642 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.273626 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4eac7522-f345-4051-ba91-e0b0c5ec07cf-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:14.273705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.273645 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac7f4bda-acd3-4768-a559-64008e6630ee-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:14.273705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.273654 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwsks\" (UniqueName: \"kubernetes.io/projected/4eac7522-f345-4051-ba91-e0b0c5ec07cf-kube-api-access-pwsks\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:14.273705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.273663 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsdl4\" (UniqueName: \"kubernetes.io/projected/ac7f4bda-acd3-4768-a559-64008e6630ee-kube-api-access-lsdl4\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:14.280828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.280807 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt"] Jul 24 22:48:14.284614 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:14.284593 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85e566ec-854fcf587d-z74nt"] Jul 24 22:48:15.177163 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:15.177127 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eac7522-f345-4051-ba91-e0b0c5ec07cf" path="/var/lib/kubelet/pods/4eac7522-f345-4051-ba91-e0b0c5ec07cf/volumes" Jul 24 22:48:15.177518 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:15.177500 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac7f4bda-acd3-4768-a559-64008e6630ee" path="/var/lib/kubelet/pods/ac7f4bda-acd3-4768-a559-64008e6630ee/volumes" Jul 24 22:48:25.000292 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:25.000259 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv"] Jul 24 22:48:25.000705 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:25.000470 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" containerID="cri-o://09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e" gracePeriod=30 Jul 24 22:48:25.270876 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:25.270797 2554 patch_prober.go:28] interesting pod/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 22:48:25.271137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:25.270867 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 22:48:30.244208 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.244186 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:30.291093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291040 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-credential-socket\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291096 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-socket\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291113 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-certs\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291135 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-data\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291183 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fktt9\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-kube-api-access-fktt9\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291204 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-podinfo\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291226 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-envoy\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291562 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291268 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istiod-ca-cert\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291562 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291320 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-token\") pod \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\" (UID: \"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209\") " Jul 24 22:48:30.291562 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291383 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:48:30.291562 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291406 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:48:30.291562 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291422 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:48:30.291823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291613 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.291823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291632 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.291823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291647 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.291823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.291639 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:48:30.292509 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.292480 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:48:30.293682 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.293662 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:48:30.293799 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.293698 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:48:30.293859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.293809 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-kube-api-access-fktt9" (OuterVolumeSpecName: "kube-api-access-fktt9") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "kube-api-access-fktt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:48:30.294124 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.294109 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" (UID: "cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:48:30.327594 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.327566 2554 generic.go:356] "Generic (PLEG): container finished" podID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerID="09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e" exitCode=0 Jul 24 22:48:30.327695 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.327614 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" Jul 24 22:48:30.327695 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.327634 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" event={"ID":"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209","Type":"ContainerDied","Data":"09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e"} Jul 24 22:48:30.327768 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.327719 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv" event={"ID":"cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209","Type":"ContainerDied","Data":"4d851be935dda9866d761ea4583038345d28674180d65461b8ae57582d4009b9"} Jul 24 22:48:30.327768 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.327758 2554 scope.go:117] "RemoveContainer" containerID="09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e" Jul 24 22:48:30.336727 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.336714 2554 scope.go:117] "RemoveContainer" containerID="17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a" Jul 24 22:48:30.344598 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.344580 2554 scope.go:117] "RemoveContainer" containerID="09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e" Jul 24 22:48:30.344855 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:48:30.344824 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e\": container with ID starting with 09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e not found: ID does not exist" containerID="09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e" Jul 24 22:48:30.344909 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.344862 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e"} err="failed to get container status \"09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e\": rpc error: code = NotFound desc = could not find container \"09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e\": container with ID starting with 09aaff5e7458606e64a3a824d562453a40bfde20bcf8fe5ea36dea01ae7bc05e not found: ID does not exist" Jul 24 22:48:30.344909 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.344879 2554 scope.go:117] "RemoveContainer" containerID="17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a" Jul 24 22:48:30.345132 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:48:30.345115 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a\": container with ID starting with 17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a not found: ID does not exist" containerID="17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a" Jul 24 22:48:30.345205 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.345140 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a"} err="failed to get container status \"17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a\": rpc error: code = NotFound desc = could not find container \"17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a\": container with ID starting with 17246567b7ad5e02a0f98410e70ba5db198fc3b7bb9734ccf5003c89b6fe375a not found: ID does not exist" Jul 24 22:48:30.348569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.348548 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv"] Jul 24 22:48:30.351974 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.351954 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-85e566ec-openshift-default-5584db7984-nnvwv"] Jul 24 22:48:30.392926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.392904 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.392926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.392923 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.393045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.392931 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.393045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.392939 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fktt9\" (UniqueName: \"kubernetes.io/projected/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-kube-api-access-fktt9\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.393045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.392948 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:30.393045 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:30.392956 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:48:31.176322 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:31.176282 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" path="/var/lib/kubelet/pods/cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209/volumes" Jul 24 22:48:44.111313 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111274 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl"] Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111582 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac7f4bda-acd3-4768-a559-64008e6630ee" containerName="payload-pre-processing" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111594 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac7f4bda-acd3-4768-a559-64008e6630ee" containerName="payload-pre-processing" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111607 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111612 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111620 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eac7522-f345-4051-ba91-e0b0c5ec07cf" containerName="payload-processing" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111626 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eac7522-f345-4051-ba91-e0b0c5ec07cf" containerName="payload-processing" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111643 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111649 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111693 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111701 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac7f4bda-acd3-4768-a559-64008e6630ee" containerName="payload-pre-processing" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111705 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbf7f6cc-7ab1-4e88-9aa8-24a61c7e9209" containerName="istio-proxy" Jul 24 22:48:44.111709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.111712 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eac7522-f345-4051-ba91-e0b0c5ec07cf" containerName="payload-processing" Jul 24 22:48:44.113838 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.113821 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.116298 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.116280 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-540d8b1d-openshift-default-dockercfg-z6hbw\"" Jul 24 22:48:44.127626 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.127601 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl"] Jul 24 22:48:44.192830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.192790 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-socket\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.192830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.192826 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfqrl\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-kube-api-access-nfqrl\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.192849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65af6821-247d-421b-9576-0bae69d809c1-istiod-ca-cert\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.192921 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-certs\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.192957 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-data\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.192990 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-credential-socket\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.193014 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65af6821-247d-421b-9576-0bae69d809c1-istio-podinfo\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.193031 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-envoy\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.193287 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.193075 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-istio-token\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294295 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65af6821-247d-421b-9576-0bae69d809c1-istio-podinfo\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294332 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-envoy\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294348 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-istio-token\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294371 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-socket\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfqrl\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-kube-api-access-nfqrl\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294406 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65af6821-247d-421b-9576-0bae69d809c1-istiod-ca-cert\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294552 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-certs\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294852 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294609 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-data\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294852 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294680 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-credential-socket\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294860 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-socket\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.294948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294897 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-certs\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.295019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.294957 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-data\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.295212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.295195 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65af6821-247d-421b-9576-0bae69d809c1-istiod-ca-cert\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.295268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.295246 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-credential-socket\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.296700 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.296678 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-envoy\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.296977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.296961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65af6821-247d-421b-9576-0bae69d809c1-istio-podinfo\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.302176 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.302133 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-istio-token\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.302552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.302532 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfqrl\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-kube-api-access-nfqrl\") pod \"e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.423428 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.423393 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:44.561983 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.561959 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl"] Jul 24 22:48:44.564089 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:48:44.564041 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65af6821_247d_421b_9576_0bae69d809c1.slice/crio-823ebff09ea5faf84f84a5843b95a5c648ee7c031785a55f9a89096fdf405603 WatchSource:0}: Error finding container 823ebff09ea5faf84f84a5843b95a5c648ee7c031785a55f9a89096fdf405603: Status 404 returned error can't find the container with id 823ebff09ea5faf84f84a5843b95a5c648ee7c031785a55f9a89096fdf405603 Jul 24 22:48:44.566283 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.566247 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:48:44.566368 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.566314 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:48:44.566368 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:44.566343 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:48:45.401797 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:45.401758 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" event={"ID":"65af6821-247d-421b-9576-0bae69d809c1","Type":"ContainerStarted","Data":"98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80"} Jul 24 22:48:45.402222 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:45.401817 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" event={"ID":"65af6821-247d-421b-9576-0bae69d809c1","Type":"ContainerStarted","Data":"823ebff09ea5faf84f84a5843b95a5c648ee7c031785a55f9a89096fdf405603"} Jul 24 22:48:45.422798 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:45.422744 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" podStartSLOduration=1.422725824 podStartE2EDuration="1.422725824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:48:45.421322158 +0000 UTC m=+2022.808140204" watchObservedRunningTime="2026-07-24 22:48:45.422725824 +0000 UTC m=+2022.809543870" Jul 24 22:48:45.424170 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:45.424134 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:45.429372 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:45.429351 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:46.406470 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:46.406432 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:46.407429 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:46.407411 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:48:48.102258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.102227 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp"] Jul 24 22:48:48.104627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.104612 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.108388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.108364 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-540d8b1d-dockercfg-fqbgs\"" Jul 24 22:48:48.108604 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.108589 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-540d8b1d\"" Jul 24 22:48:48.116096 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.116075 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk"] Jul 24 22:48:48.118346 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.118327 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.120697 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.120673 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp"] Jul 24 22:48:48.132063 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.132019 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk"] Jul 24 22:48:48.226683 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.226648 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5td8\" (UniqueName: \"kubernetes.io/projected/92097b66-ffba-4275-957d-66c8810a3189-kube-api-access-s5td8\") pod \"payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.226839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.226710 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/473c4c00-dccf-404b-96c0-350cc2789b41-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.226839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.226815 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ggmr\" (UniqueName: \"kubernetes.io/projected/473c4c00-dccf-404b-96c0-350cc2789b41-kube-api-access-4ggmr\") pod \"payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.226909 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.226840 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92097b66-ffba-4275-957d-66c8810a3189-plugins-config-volume\") pod \"payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.327695 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.327668 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5td8\" (UniqueName: \"kubernetes.io/projected/92097b66-ffba-4275-957d-66c8810a3189-kube-api-access-s5td8\") pod \"payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.327842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.327699 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/473c4c00-dccf-404b-96c0-350cc2789b41-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.327842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.327760 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ggmr\" (UniqueName: \"kubernetes.io/projected/473c4c00-dccf-404b-96c0-350cc2789b41-kube-api-access-4ggmr\") pod \"payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.327842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.327782 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92097b66-ffba-4275-957d-66c8810a3189-plugins-config-volume\") pod \"payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.328497 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.328478 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92097b66-ffba-4275-957d-66c8810a3189-plugins-config-volume\") pod \"payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.328582 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.328482 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/473c4c00-dccf-404b-96c0-350cc2789b41-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.337952 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.337928 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5td8\" (UniqueName: \"kubernetes.io/projected/92097b66-ffba-4275-957d-66c8810a3189-kube-api-access-s5td8\") pod \"payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.338046 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.337961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ggmr\" (UniqueName: \"kubernetes.io/projected/473c4c00-dccf-404b-96c0-350cc2789b41-kube-api-access-4ggmr\") pod \"payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.415828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.415807 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:48.427374 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.427337 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:48.561560 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.561530 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp"] Jul 24 22:48:48.564572 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:48:48.564530 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod473c4c00_dccf_404b_96c0_350cc2789b41.slice/crio-623a65e8da2725cfbb48cc9af79f55d317585a2d710f4060ec7d9370353a8ab4 WatchSource:0}: Error finding container 623a65e8da2725cfbb48cc9af79f55d317585a2d710f4060ec7d9370353a8ab4: Status 404 returned error can't find the container with id 623a65e8da2725cfbb48cc9af79f55d317585a2d710f4060ec7d9370353a8ab4 Jul 24 22:48:48.591266 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:48.591247 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk"] Jul 24 22:48:48.602313 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:48:48.602292 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92097b66_ffba_4275_957d_66c8810a3189.slice/crio-bc8e75a542deab202edc80bcf949246a3c796b1db118a040f1485acea6cf0c65 WatchSource:0}: Error finding container bc8e75a542deab202edc80bcf949246a3c796b1db118a040f1485acea6cf0c65: Status 404 returned error can't find the container with id bc8e75a542deab202edc80bcf949246a3c796b1db118a040f1485acea6cf0c65 Jul 24 22:48:49.427671 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.427630 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" event={"ID":"92097b66-ffba-4275-957d-66c8810a3189","Type":"ContainerStarted","Data":"2a19015237afdb2b14e1009dce8ca2825b840daa1284c3d076cce3eeb61dfe60"} Jul 24 22:48:49.428133 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.427723 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" event={"ID":"92097b66-ffba-4275-957d-66c8810a3189","Type":"ContainerStarted","Data":"bc8e75a542deab202edc80bcf949246a3c796b1db118a040f1485acea6cf0c65"} Jul 24 22:48:49.428133 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.427752 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:48:49.429248 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.429227 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" event={"ID":"473c4c00-dccf-404b-96c0-350cc2789b41","Type":"ContainerStarted","Data":"957b14ab017ebe04206eb9e882d2852c40ebeebc942812927160e5dbb4cdad6d"} Jul 24 22:48:49.429342 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.429263 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" event={"ID":"473c4c00-dccf-404b-96c0-350cc2789b41","Type":"ContainerStarted","Data":"623a65e8da2725cfbb48cc9af79f55d317585a2d710f4060ec7d9370353a8ab4"} Jul 24 22:48:49.429383 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.429354 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:48:49.444828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.444780 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" podStartSLOduration=1.444768746 podStartE2EDuration="1.444768746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:48:49.442848788 +0000 UTC m=+2026.829666832" watchObservedRunningTime="2026-07-24 22:48:49.444768746 +0000 UTC m=+2026.831586793" Jul 24 22:48:49.457545 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:48:49.457505 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" podStartSLOduration=1.457494524 podStartE2EDuration="1.457494524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:48:49.456741459 +0000 UTC m=+2026.843559506" watchObservedRunningTime="2026-07-24 22:48:49.457494524 +0000 UTC m=+2026.844312569" Jul 24 22:49:00.435782 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.435758 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:49:00.436226 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.435820 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:49:00.491130 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.491101 2554 generic.go:356] "Generic (PLEG): container finished" podID="65af6821-247d-421b-9576-0bae69d809c1" containerID="98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80" exitCode=0 Jul 24 22:49:00.491399 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.491140 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" event={"ID":"65af6821-247d-421b-9576-0bae69d809c1","Type":"ContainerDied","Data":"98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80"} Jul 24 22:49:00.491568 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.491550 2554 scope.go:117] "RemoveContainer" containerID="98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80" Jul 24 22:49:00.492676 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.492647 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:49:00.492784 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.492721 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:49:00.492784 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:00.492765 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:49:01.406623 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:01.406587 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:49:01.497980 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:01.497951 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" event={"ID":"65af6821-247d-421b-9576-0bae69d809c1","Type":"ContainerStarted","Data":"f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4"} Jul 24 22:49:02.423884 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:02.423846 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:49:02.425083 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:02.425057 2554 patch_prober.go:28] interesting pod/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 24 22:49:02.425234 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:02.425139 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 24 22:49:03.424198 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:03.424134 2554 patch_prober.go:28] interesting pod/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 24 22:49:03.424539 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:03.424216 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 24 22:49:04.424186 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:04.424131 2554 patch_prober.go:28] interesting pod/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 24 22:49:04.424354 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:04.424215 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 24 22:49:05.424448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:05.424413 2554 patch_prober.go:28] interesting pod/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 24 22:49:05.424842 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:05.424475 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 24 22:49:06.427622 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:06.427592 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:49:06.428117 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:06.427966 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:49:06.428576 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:06.428557 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:49:51.500063 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.500020 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk"] Jul 24 22:49:51.500598 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.500287 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" podUID="92097b66-ffba-4275-957d-66c8810a3189" containerName="payload-processing" containerID="cri-o://2a19015237afdb2b14e1009dce8ca2825b840daa1284c3d076cce3eeb61dfe60" gracePeriod=30 Jul 24 22:49:51.506764 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.506726 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp"] Jul 24 22:49:51.507205 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.507015 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" podUID="473c4c00-dccf-404b-96c0-350cc2789b41" containerName="payload-pre-processing" containerID="cri-o://957b14ab017ebe04206eb9e882d2852c40ebeebc942812927160e5dbb4cdad6d" gracePeriod=30 Jul 24 22:49:51.740608 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.740570 2554 generic.go:356] "Generic (PLEG): container finished" podID="92097b66-ffba-4275-957d-66c8810a3189" containerID="2a19015237afdb2b14e1009dce8ca2825b840daa1284c3d076cce3eeb61dfe60" exitCode=0 Jul 24 22:49:51.740775 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.740647 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" event={"ID":"92097b66-ffba-4275-957d-66c8810a3189","Type":"ContainerDied","Data":"2a19015237afdb2b14e1009dce8ca2825b840daa1284c3d076cce3eeb61dfe60"} Jul 24 22:49:51.742822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.742695 2554 generic.go:356] "Generic (PLEG): container finished" podID="473c4c00-dccf-404b-96c0-350cc2789b41" containerID="957b14ab017ebe04206eb9e882d2852c40ebeebc942812927160e5dbb4cdad6d" exitCode=0 Jul 24 22:49:51.742822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.742741 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" event={"ID":"473c4c00-dccf-404b-96c0-350cc2789b41","Type":"ContainerDied","Data":"957b14ab017ebe04206eb9e882d2852c40ebeebc942812927160e5dbb4cdad6d"} Jul 24 22:49:51.761244 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.761187 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:49:51.794204 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.794179 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:49:51.806379 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.806358 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5td8\" (UniqueName: \"kubernetes.io/projected/92097b66-ffba-4275-957d-66c8810a3189-kube-api-access-s5td8\") pod \"92097b66-ffba-4275-957d-66c8810a3189\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " Jul 24 22:49:51.806470 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.806410 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92097b66-ffba-4275-957d-66c8810a3189-plugins-config-volume\") pod \"92097b66-ffba-4275-957d-66c8810a3189\" (UID: \"92097b66-ffba-4275-957d-66c8810a3189\") " Jul 24 22:49:51.806792 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.806764 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92097b66-ffba-4275-957d-66c8810a3189-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "92097b66-ffba-4275-957d-66c8810a3189" (UID: "92097b66-ffba-4275-957d-66c8810a3189"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:49:51.808481 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.808459 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92097b66-ffba-4275-957d-66c8810a3189-kube-api-access-s5td8" (OuterVolumeSpecName: "kube-api-access-s5td8") pod "92097b66-ffba-4275-957d-66c8810a3189" (UID: "92097b66-ffba-4275-957d-66c8810a3189"). InnerVolumeSpecName "kube-api-access-s5td8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:49:51.907760 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.907714 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/473c4c00-dccf-404b-96c0-350cc2789b41-plugins-config-volume\") pod \"473c4c00-dccf-404b-96c0-350cc2789b41\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " Jul 24 22:49:51.907918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.907776 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ggmr\" (UniqueName: \"kubernetes.io/projected/473c4c00-dccf-404b-96c0-350cc2789b41-kube-api-access-4ggmr\") pod \"473c4c00-dccf-404b-96c0-350cc2789b41\" (UID: \"473c4c00-dccf-404b-96c0-350cc2789b41\") " Jul 24 22:49:51.908011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.907994 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5td8\" (UniqueName: \"kubernetes.io/projected/92097b66-ffba-4275-957d-66c8810a3189-kube-api-access-s5td8\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:49:51.908057 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.908019 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/92097b66-ffba-4275-957d-66c8810a3189-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:49:51.908093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.908071 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/473c4c00-dccf-404b-96c0-350cc2789b41-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "473c4c00-dccf-404b-96c0-350cc2789b41" (UID: "473c4c00-dccf-404b-96c0-350cc2789b41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:49:51.909936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:51.909915 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/473c4c00-dccf-404b-96c0-350cc2789b41-kube-api-access-4ggmr" (OuterVolumeSpecName: "kube-api-access-4ggmr") pod "473c4c00-dccf-404b-96c0-350cc2789b41" (UID: "473c4c00-dccf-404b-96c0-350cc2789b41"). InnerVolumeSpecName "kube-api-access-4ggmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:49:52.008847 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.008815 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/473c4c00-dccf-404b-96c0-350cc2789b41-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:49:52.008847 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.008841 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ggmr\" (UniqueName: \"kubernetes.io/projected/473c4c00-dccf-404b-96c0-350cc2789b41-kube-api-access-4ggmr\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:49:52.748282 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.748254 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" Jul 24 22:49:52.748282 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.748275 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp" event={"ID":"473c4c00-dccf-404b-96c0-350cc2789b41","Type":"ContainerDied","Data":"623a65e8da2725cfbb48cc9af79f55d317585a2d710f4060ec7d9370353a8ab4"} Jul 24 22:49:52.748788 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.748343 2554 scope.go:117] "RemoveContainer" containerID="957b14ab017ebe04206eb9e882d2852c40ebeebc942812927160e5dbb4cdad6d" Jul 24 22:49:52.749724 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.749624 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" Jul 24 22:49:52.749724 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.749670 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk" event={"ID":"92097b66-ffba-4275-957d-66c8810a3189","Type":"ContainerDied","Data":"bc8e75a542deab202edc80bcf949246a3c796b1db118a040f1485acea6cf0c65"} Jul 24 22:49:52.757677 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.757551 2554 scope.go:117] "RemoveContainer" containerID="2a19015237afdb2b14e1009dce8ca2825b840daa1284c3d076cce3eeb61dfe60" Jul 24 22:49:52.786690 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.786198 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk"] Jul 24 22:49:52.787964 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.787941 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-540d8b1d-df98989b6-kvmwk"] Jul 24 22:49:52.795246 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.795223 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp"] Jul 24 22:49:52.798345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:52.798320 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-540d8b1d-856d54688d-5vwmp"] Jul 24 22:49:53.175999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:53.175967 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="473c4c00-dccf-404b-96c0-350cc2789b41" path="/var/lib/kubelet/pods/473c4c00-dccf-404b-96c0-350cc2789b41/volumes" Jul 24 22:49:53.176359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:49:53.176345 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92097b66-ffba-4275-957d-66c8810a3189" path="/var/lib/kubelet/pods/92097b66-ffba-4275-957d-66c8810a3189/volumes" Jul 24 22:51:01.144643 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:01.144611 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl"] Jul 24 22:51:01.145050 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:01.144820 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" containerID="cri-o://f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4" gracePeriod=30 Jul 24 22:51:06.388104 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.388078 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:51:06.478183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478093 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-data\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478174 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfqrl\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-kube-api-access-nfqrl\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478192 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-envoy\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478430 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478211 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-certs\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478430 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478253 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65af6821-247d-421b-9576-0bae69d809c1-istiod-ca-cert\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478430 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478270 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65af6821-247d-421b-9576-0bae69d809c1-istio-podinfo\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478430 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478309 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-istio-token\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478630 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478443 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-socket\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478630 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478502 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-credential-socket\") pod \"65af6821-247d-421b-9576-0bae69d809c1\" (UID: \"65af6821-247d-421b-9576-0bae69d809c1\") " Jul 24 22:51:06.478776 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478682 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:51:06.478839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478813 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.479041 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478884 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:51:06.479041 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.478962 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65af6821-247d-421b-9576-0bae69d809c1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:51:06.479041 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.479035 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-data" (OuterVolumeSpecName: "istio-data") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:51:06.479383 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.479274 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:51:06.480612 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.480583 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:51:06.480715 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.480659 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65af6821-247d-421b-9576-0bae69d809c1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:51:06.480893 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.480878 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-kube-api-access-nfqrl" (OuterVolumeSpecName: "kube-api-access-nfqrl") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "kube-api-access-nfqrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:51:06.481001 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.480981 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-istio-token" (OuterVolumeSpecName: "istio-token") pod "65af6821-247d-421b-9576-0bae69d809c1" (UID: "65af6821-247d-421b-9576-0bae69d809c1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:51:06.580239 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580214 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580239 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580234 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580244 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580253 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580261 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfqrl\" (UniqueName: \"kubernetes.io/projected/65af6821-247d-421b-9576-0bae69d809c1-kube-api-access-nfqrl\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580269 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65af6821-247d-421b-9576-0bae69d809c1-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580278 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65af6821-247d-421b-9576-0bae69d809c1-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:06.580370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:06.580285 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65af6821-247d-421b-9576-0bae69d809c1-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:51:07.099317 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.099286 2554 generic.go:356] "Generic (PLEG): container finished" podID="65af6821-247d-421b-9576-0bae69d809c1" containerID="f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4" exitCode=0 Jul 24 22:51:07.099516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.099346 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" event={"ID":"65af6821-247d-421b-9576-0bae69d809c1","Type":"ContainerDied","Data":"f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4"} Jul 24 22:51:07.099516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.099347 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" Jul 24 22:51:07.099516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.099389 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl" event={"ID":"65af6821-247d-421b-9576-0bae69d809c1","Type":"ContainerDied","Data":"823ebff09ea5faf84f84a5843b95a5c648ee7c031785a55f9a89096fdf405603"} Jul 24 22:51:07.099516 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.099410 2554 scope.go:117] "RemoveContainer" containerID="f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4" Jul 24 22:51:07.109268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.109252 2554 scope.go:117] "RemoveContainer" containerID="98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80" Jul 24 22:51:07.117051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.117035 2554 scope.go:117] "RemoveContainer" containerID="f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4" Jul 24 22:51:07.117310 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:51:07.117292 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4\": container with ID starting with f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4 not found: ID does not exist" containerID="f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4" Jul 24 22:51:07.117366 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.117318 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4"} err="failed to get container status \"f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4\": rpc error: code = NotFound desc = could not find container \"f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4\": container with ID starting with f95dc36c8d38a81134168c132d9090408104ada01e73dfa7af9016a6ef4535f4 not found: ID does not exist" Jul 24 22:51:07.117366 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.117334 2554 scope.go:117] "RemoveContainer" containerID="98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80" Jul 24 22:51:07.117587 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:51:07.117569 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80\": container with ID starting with 98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80 not found: ID does not exist" containerID="98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80" Jul 24 22:51:07.117627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.117596 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80"} err="failed to get container status \"98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80\": rpc error: code = NotFound desc = could not find container \"98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80\": container with ID starting with 98d65ec07952aea3e093697206bb53e5300a7cd9162271b1dadcf5f9b0890a80 not found: ID does not exist" Jul 24 22:51:07.122305 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.122284 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl"] Jul 24 22:51:07.127456 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.127437 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-540d8b1d-openshift-default-665848bfd8-2klrl"] Jul 24 22:51:07.175360 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:07.175339 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65af6821-247d-421b-9576-0bae69d809c1" path="/var/lib/kubelet/pods/65af6821-247d-421b-9576-0bae69d809c1/volumes" Jul 24 22:51:56.634475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634436 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg"] Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634721 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634733 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634749 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92097b66-ffba-4275-957d-66c8810a3189" containerName="payload-processing" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634755 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="92097b66-ffba-4275-957d-66c8810a3189" containerName="payload-processing" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634768 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634774 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634780 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="473c4c00-dccf-404b-96c0-350cc2789b41" containerName="payload-pre-processing" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634786 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="473c4c00-dccf-404b-96c0-350cc2789b41" containerName="payload-pre-processing" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634833 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="92097b66-ffba-4275-957d-66c8810a3189" containerName="payload-processing" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634842 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="473c4c00-dccf-404b-96c0-350cc2789b41" containerName="payload-pre-processing" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634850 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" Jul 24 22:51:56.634977 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.634856 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="65af6821-247d-421b-9576-0bae69d809c1" containerName="istio-proxy" Jul 24 22:51:56.637680 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.637664 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.640389 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.640364 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-11557e-openshift-default-dockercfg-svxdk\"" Jul 24 22:51:56.649634 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.649609 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg"] Jul 24 22:51:56.660561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660314 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-data\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660365 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-credential-socket\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660395 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istiod-ca-cert\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660423 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-token\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660454 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-socket\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660561 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660536 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-certs\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660884 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-envoy\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660884 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660612 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-podinfo\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.660884 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.660687 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj5jw\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-kube-api-access-pj5jw\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761515 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761475 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-envoy\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761515 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761519 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-podinfo\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761651 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5jw\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-kube-api-access-pj5jw\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-data\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761723 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-credential-socket\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761743 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istiod-ca-cert\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.761766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761765 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-token\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762042 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761798 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-socket\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762042 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.761883 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-certs\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.762110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-credential-socket\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762247 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.762140 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-data\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762247 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.762203 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-certs\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762519 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.762362 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-socket\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.762603 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.762582 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istiod-ca-cert\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.763949 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.763921 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-envoy\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.764188 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.764172 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-podinfo\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.770458 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.770420 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-token\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.770571 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.770483 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5jw\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-kube-api-access-pj5jw\") pod \"e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:56.949108 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:56.949075 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:57.088490 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.088445 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg"] Jul 24 22:51:57.092698 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.092654 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:51:57.092830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.092796 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:51:57.092915 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.092891 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:51:57.339552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.339473 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" event={"ID":"6e9c8ab8-1eca-4563-a699-c04b09e4a379","Type":"ContainerStarted","Data":"aaed2fc940b43d0ef74b172724b9996195d19a38bace6c8e15eeec5564e9f49b"} Jul 24 22:51:57.339552 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.339523 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" event={"ID":"6e9c8ab8-1eca-4563-a699-c04b09e4a379","Type":"ContainerStarted","Data":"308989a18627de3636f8919c22eb982a27d051c8b799235484dafb9e3d384399"} Jul 24 22:51:57.362467 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.362423 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podStartSLOduration=1.362408937 podStartE2EDuration="1.362408937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:51:57.36103043 +0000 UTC m=+2214.747848478" watchObservedRunningTime="2026-07-24 22:51:57.362408937 +0000 UTC m=+2214.749226983" Jul 24 22:51:57.949435 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:57.949403 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:58.955300 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:58.955265 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:59.349346 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:59.349266 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:51:59.350345 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:51:59.350318 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:52:00.012640 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.012599 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp"] Jul 24 22:52:00.015961 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.015943 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.018568 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.018544 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-11557e\"" Jul 24 22:52:00.018706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.018544 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-11557e-dockercfg-z95g4\"" Jul 24 22:52:00.023423 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.023397 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp"] Jul 24 22:52:00.044407 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.044375 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645"] Jul 24 22:52:00.047681 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.047655 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.059550 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.059526 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645"] Jul 24 22:52:00.091654 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.091623 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62e21225-27d7-4776-8736-c0bb300956f7-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.091839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.091666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwk29\" (UniqueName: \"kubernetes.io/projected/9caea192-ffe7-466c-8c2e-496ced3fc700-kube-api-access-hwk29\") pod \"payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.091839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.091741 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24mjd\" (UniqueName: \"kubernetes.io/projected/62e21225-27d7-4776-8736-c0bb300956f7-kube-api-access-24mjd\") pod \"payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.091839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.091810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9caea192-ffe7-466c-8c2e-496ced3fc700-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.193089 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.193055 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwk29\" (UniqueName: \"kubernetes.io/projected/9caea192-ffe7-466c-8c2e-496ced3fc700-kube-api-access-hwk29\") pod \"payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.193303 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.193111 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24mjd\" (UniqueName: \"kubernetes.io/projected/62e21225-27d7-4776-8736-c0bb300956f7-kube-api-access-24mjd\") pod \"payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.193376 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.193318 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9caea192-ffe7-466c-8c2e-496ced3fc700-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.193586 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.193564 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62e21225-27d7-4776-8736-c0bb300956f7-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.194291 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.194267 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9caea192-ffe7-466c-8c2e-496ced3fc700-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.194364 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.194333 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62e21225-27d7-4776-8736-c0bb300956f7-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.202714 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.202693 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24mjd\" (UniqueName: \"kubernetes.io/projected/62e21225-27d7-4776-8736-c0bb300956f7-kube-api-access-24mjd\") pod \"payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.203203 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.203187 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwk29\" (UniqueName: \"kubernetes.io/projected/9caea192-ffe7-466c-8c2e-496ced3fc700-kube-api-access-hwk29\") pod \"payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.328851 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.328790 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:00.358228 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.358204 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:00.486384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.486324 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp"] Jul 24 22:52:00.488452 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:52:00.488425 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9caea192_ffe7_466c_8c2e_496ced3fc700.slice/crio-0268d5a2380bb979e703753e795c88326d85b15234afb0a0ae2a31664abec5ae WatchSource:0}: Error finding container 0268d5a2380bb979e703753e795c88326d85b15234afb0a0ae2a31664abec5ae: Status 404 returned error can't find the container with id 0268d5a2380bb979e703753e795c88326d85b15234afb0a0ae2a31664abec5ae Jul 24 22:52:00.502798 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:00.502779 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645"] Jul 24 22:52:00.505201 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:52:00.505180 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62e21225_27d7_4776_8736_c0bb300956f7.slice/crio-518a1076aef9f8f95f036d3faa9cb10e63051b59059fb1394eb143c3c74f9796 WatchSource:0}: Error finding container 518a1076aef9f8f95f036d3faa9cb10e63051b59059fb1394eb143c3c74f9796: Status 404 returned error can't find the container with id 518a1076aef9f8f95f036d3faa9cb10e63051b59059fb1394eb143c3c74f9796 Jul 24 22:52:01.363297 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.363142 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" event={"ID":"9caea192-ffe7-466c-8c2e-496ced3fc700","Type":"ContainerStarted","Data":"7a64be530c4c4bb6d444184b097261982c05ec23c0a87cabec432c6a9edcad88"} Jul 24 22:52:01.363736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.363330 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:01.363736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.363355 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" event={"ID":"9caea192-ffe7-466c-8c2e-496ced3fc700","Type":"ContainerStarted","Data":"0268d5a2380bb979e703753e795c88326d85b15234afb0a0ae2a31664abec5ae"} Jul 24 22:52:01.364732 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.364709 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" event={"ID":"62e21225-27d7-4776-8736-c0bb300956f7","Type":"ContainerStarted","Data":"ab0aca56205a3a99715264676346dc41bfa50ba1aaef1b55c0c5f83e8b7346ee"} Jul 24 22:52:01.364828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.364750 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" event={"ID":"62e21225-27d7-4776-8736-c0bb300956f7","Type":"ContainerStarted","Data":"518a1076aef9f8f95f036d3faa9cb10e63051b59059fb1394eb143c3c74f9796"} Jul 24 22:52:01.364948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.364937 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:01.383850 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.383799 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" podStartSLOduration=2.383788202 podStartE2EDuration="2.383788202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:01.38240215 +0000 UTC m=+2218.769220196" watchObservedRunningTime="2026-07-24 22:52:01.383788202 +0000 UTC m=+2218.770606247" Jul 24 22:52:01.399047 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:01.399000 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" podStartSLOduration=1.398988382 podStartE2EDuration="1.398988382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:01.398076646 +0000 UTC m=+2218.784894719" watchObservedRunningTime="2026-07-24 22:52:01.398988382 +0000 UTC m=+2218.785806428" Jul 24 22:52:04.312729 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.312693 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266"] Jul 24 22:52:04.317384 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.317363 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.320106 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.320082 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-422c4e-openshift-default-dockercfg-sm2g7\"" Jul 24 22:52:04.327388 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.327361 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266"] Jul 24 22:52:04.428672 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428642 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-envoy\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428672 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428676 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-data\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428693 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-token\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428718 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-credential-socket\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428766 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istiod-ca-cert\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428809 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-socket\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428866 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-podinfo\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.428894 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428884 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbjl5\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-kube-api-access-fbjl5\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.429094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.428912 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-certs\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530269 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530221 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-envoy\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530284 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-data\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530418 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-token\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530471 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-credential-socket\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530492 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istiod-ca-cert\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530588 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530532 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-socket\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530607 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-podinfo\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530634 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbjl5\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-kube-api-access-fbjl5\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530680 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-certs\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530754 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530699 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-data\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.530967 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.530845 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-credential-socket\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.531216 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.531173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-socket\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.531335 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.531290 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-certs\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.531408 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.531372 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istiod-ca-cert\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.533538 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.533514 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-envoy\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.533724 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.533696 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-podinfo\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.539505 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.539474 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-token\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.539640 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.539603 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbjl5\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-kube-api-access-fbjl5\") pod \"e2e-inf-b-422c4e-openshift-default-7585b75796-5x266\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.628401 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.628308 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:04.780393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.780368 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266"] Jul 24 22:52:04.781646 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:52:04.781625 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod393a24aa_2ccb_43f7_8856_9a0fd126ee01.slice/crio-49b733be65f943b7b04cd6a549a21109254e504af14f8864b6d90790eec24c28 WatchSource:0}: Error finding container 49b733be65f943b7b04cd6a549a21109254e504af14f8864b6d90790eec24c28: Status 404 returned error can't find the container with id 49b733be65f943b7b04cd6a549a21109254e504af14f8864b6d90790eec24c28 Jul 24 22:52:04.783868 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.783835 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:04.783976 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.783910 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:04.783976 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:04.783952 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:05.387063 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:05.387020 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" event={"ID":"393a24aa-2ccb-43f7-8856-9a0fd126ee01","Type":"ContainerStarted","Data":"8f57462014a768813fe618412ba81ba27a2a47277a61aa26052b785bf82be778"} Jul 24 22:52:05.387462 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:05.387087 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" event={"ID":"393a24aa-2ccb-43f7-8856-9a0fd126ee01","Type":"ContainerStarted","Data":"49b733be65f943b7b04cd6a549a21109254e504af14f8864b6d90790eec24c28"} Jul 24 22:52:05.413798 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:05.413745 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" podStartSLOduration=1.413729678 podStartE2EDuration="1.413729678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:05.411185849 +0000 UTC m=+2222.798003917" watchObservedRunningTime="2026-07-24 22:52:05.413729678 +0000 UTC m=+2222.800547723" Jul 24 22:52:05.629070 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:05.629035 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:05.634101 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:05.634077 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:06.392525 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:06.392497 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:06.393680 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:06.393656 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:08.274178 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.273872 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl"] Jul 24 22:52:08.278993 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.278970 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.281859 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.281835 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-422c4e\"" Jul 24 22:52:08.281984 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.281866 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-422c4e-dockercfg-z7qv4\"" Jul 24 22:52:08.285940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.285912 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl"] Jul 24 22:52:08.310916 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.310894 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz"] Jul 24 22:52:08.314628 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.314603 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.324426 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.324405 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz"] Jul 24 22:52:08.375645 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.375607 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbpd\" (UniqueName: \"kubernetes.io/projected/351d7d76-8482-43ab-9f8e-a4521be8003f-kube-api-access-txbpd\") pod \"payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.375940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.375916 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g6cq\" (UniqueName: \"kubernetes.io/projected/5b915f38-68d1-41d6-a026-3dfacd317c92-kube-api-access-4g6cq\") pod \"payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.376212 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.376192 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b915f38-68d1-41d6-a026-3dfacd317c92-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.376396 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.376378 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351d7d76-8482-43ab-9f8e-a4521be8003f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.477318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.477288 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g6cq\" (UniqueName: \"kubernetes.io/projected/5b915f38-68d1-41d6-a026-3dfacd317c92-kube-api-access-4g6cq\") pod \"payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.477499 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.477364 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b915f38-68d1-41d6-a026-3dfacd317c92-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.477499 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.477394 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351d7d76-8482-43ab-9f8e-a4521be8003f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.477499 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.477473 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbpd\" (UniqueName: \"kubernetes.io/projected/351d7d76-8482-43ab-9f8e-a4521be8003f-kube-api-access-txbpd\") pod \"payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.478268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.478246 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b915f38-68d1-41d6-a026-3dfacd317c92-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.478396 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.478368 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351d7d76-8482-43ab-9f8e-a4521be8003f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.487070 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.487042 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbpd\" (UniqueName: \"kubernetes.io/projected/351d7d76-8482-43ab-9f8e-a4521be8003f-kube-api-access-txbpd\") pod \"payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.487199 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.487130 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g6cq\" (UniqueName: \"kubernetes.io/projected/5b915f38-68d1-41d6-a026-3dfacd317c92-kube-api-access-4g6cq\") pod \"payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.591861 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.591792 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:08.626379 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.626357 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:08.772175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.769990 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl"] Jul 24 22:52:08.841079 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:08.841057 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz"] Jul 24 22:52:08.843635 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:52:08.843582 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b915f38_68d1_41d6_a026_3dfacd317c92.slice/crio-8783b52b04fea2c035f2ed553530031a547bc2cef822d57fe9b814d7bc1ca5ab WatchSource:0}: Error finding container 8783b52b04fea2c035f2ed553530031a547bc2cef822d57fe9b814d7bc1ca5ab: Status 404 returned error can't find the container with id 8783b52b04fea2c035f2ed553530031a547bc2cef822d57fe9b814d7bc1ca5ab Jul 24 22:52:09.417976 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.417933 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" event={"ID":"5b915f38-68d1-41d6-a026-3dfacd317c92","Type":"ContainerStarted","Data":"a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38"} Jul 24 22:52:09.418448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.418036 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:09.418448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.418062 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" event={"ID":"5b915f38-68d1-41d6-a026-3dfacd317c92","Type":"ContainerStarted","Data":"8783b52b04fea2c035f2ed553530031a547bc2cef822d57fe9b814d7bc1ca5ab"} Jul 24 22:52:09.419452 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.419426 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" event={"ID":"351d7d76-8482-43ab-9f8e-a4521be8003f","Type":"ContainerStarted","Data":"4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603"} Jul 24 22:52:09.419586 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.419463 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" event={"ID":"351d7d76-8482-43ab-9f8e-a4521be8003f","Type":"ContainerStarted","Data":"e06efddc79b9701375112df5125a4f3ec7062c6bc6e98061aeb4654e86aad930"} Jul 24 22:52:09.419586 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.419497 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:09.433822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.433770 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" podStartSLOduration=1.433752364 podStartE2EDuration="1.433752364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:09.432917535 +0000 UTC m=+2226.819735585" watchObservedRunningTime="2026-07-24 22:52:09.433752364 +0000 UTC m=+2226.820570411" Jul 24 22:52:09.448261 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:09.448203 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" podStartSLOduration=1.448188828 podStartE2EDuration="1.448188828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:52:09.446600696 +0000 UTC m=+2226.833418743" watchObservedRunningTime="2026-07-24 22:52:09.448188828 +0000 UTC m=+2226.835006874" Jul 24 22:52:12.375400 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.374349 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:52:12.376382 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.376083 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:52:12.439239 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.439211 2554 generic.go:356] "Generic (PLEG): container finished" podID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerID="aaed2fc940b43d0ef74b172724b9996195d19a38bace6c8e15eeec5564e9f49b" exitCode=0 Jul 24 22:52:12.439445 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.439292 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" event={"ID":"6e9c8ab8-1eca-4563-a699-c04b09e4a379","Type":"ContainerDied","Data":"aaed2fc940b43d0ef74b172724b9996195d19a38bace6c8e15eeec5564e9f49b"} Jul 24 22:52:12.439708 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.439679 2554 scope.go:117] "RemoveContainer" containerID="aaed2fc940b43d0ef74b172724b9996195d19a38bace6c8e15eeec5564e9f49b" Jul 24 22:52:12.440735 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.440708 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:12.440790 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.440775 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:12.440834 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:12.440805 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:13.446991 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:13.446958 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" event={"ID":"6e9c8ab8-1eca-4563-a699-c04b09e4a379","Type":"ContainerStarted","Data":"952347e31603275b4cdbb319945894acb3a57f9cfd5fb956541691d040b9d827"} Jul 24 22:52:13.950106 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:13.950071 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:52:13.951639 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:13.951591 2554 patch_prober.go:28] interesting pod/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 22:52:13.951816 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:13.951664 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 22:52:14.349748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:14.349651 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:52:14.950587 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:14.950537 2554 patch_prober.go:28] interesting pod/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 22:52:14.951104 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:14.950609 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 22:52:15.949748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:15.949717 2554 patch_prober.go:28] interesting pod/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 22:52:15.949943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:15.949769 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 22:52:16.950447 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:16.950410 2554 patch_prober.go:28] interesting pod/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 22:52:16.951491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:16.951045 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 22:52:17.949699 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:17.949664 2554 patch_prober.go:28] interesting pod/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 24 22:52:17.949903 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:17.949718 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 24 22:52:18.954093 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:18.954062 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:52:18.954949 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:18.954929 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:52:20.427566 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.427540 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:52:20.428074 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.427617 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:52:20.493299 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.493270 2554 generic.go:356] "Generic (PLEG): container finished" podID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerID="8f57462014a768813fe618412ba81ba27a2a47277a61aa26052b785bf82be778" exitCode=0 Jul 24 22:52:20.493299 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.493313 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" event={"ID":"393a24aa-2ccb-43f7-8856-9a0fd126ee01","Type":"ContainerDied","Data":"8f57462014a768813fe618412ba81ba27a2a47277a61aa26052b785bf82be778"} Jul 24 22:52:20.493731 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.493693 2554 scope.go:117] "RemoveContainer" containerID="8f57462014a768813fe618412ba81ba27a2a47277a61aa26052b785bf82be778" Jul 24 22:52:20.494874 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.494846 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:20.494942 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.494923 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:20.494979 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:20.494962 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:52:21.392720 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:21.392690 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:21.500823 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:21.500785 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" event={"ID":"393a24aa-2ccb-43f7-8856-9a0fd126ee01","Type":"ContainerStarted","Data":"981bca4da5e4efb20ec61610ce031774b45927a549db4610ca768ece59b50f71"} Jul 24 22:52:21.629258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:21.629225 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:21.630446 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:21.630426 2554 patch_prober.go:28] interesting pod/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 22:52:21.630518 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:21.630472 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 22:52:22.629132 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:22.629101 2554 patch_prober.go:28] interesting pod/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 22:52:22.629523 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:22.629166 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 22:52:23.629439 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:23.629363 2554 patch_prober.go:28] interesting pod/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 22:52:23.629439 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:23.629416 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 22:52:24.628971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:24.628936 2554 patch_prober.go:28] interesting pod/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 24 22:52:24.629174 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:24.628994 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 24 22:52:25.632667 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:25.632640 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:25.633060 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:25.632914 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:52:25.633581 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:52:25.633562 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:53:42.450502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.450470 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk"] Jul 24 22:53:42.453830 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.453814 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:42.456398 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.456372 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 22:53:42.456512 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.456406 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcbdx\"" Jul 24 22:53:42.460965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.460925 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk"] Jul 24 22:53:42.520431 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.520398 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e0ca81-7d76-446b-b0fb-d03faceec4d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk\" (UID: \"18e0ca81-7d76-446b-b0fb-d03faceec4d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:42.621076 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.621053 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e0ca81-7d76-446b-b0fb-d03faceec4d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk\" (UID: \"18e0ca81-7d76-446b-b0fb-d03faceec4d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:42.621611 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.621594 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e0ca81-7d76-446b-b0fb-d03faceec4d5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk\" (UID: \"18e0ca81-7d76-446b-b0fb-d03faceec4d5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:42.766095 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.766037 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:42.895445 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.895423 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk"] Jul 24 22:53:42.897531 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:53:42.897500 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e0ca81_7d76_446b_b0fb_d03faceec4d5.slice/crio-e15edee2938ae732dd001b53804e283f32388f048dc1a008b4962482da939bb5 WatchSource:0}: Error finding container e15edee2938ae732dd001b53804e283f32388f048dc1a008b4962482da939bb5: Status 404 returned error can't find the container with id e15edee2938ae732dd001b53804e283f32388f048dc1a008b4962482da939bb5 Jul 24 22:53:42.913800 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:42.913775 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" event={"ID":"18e0ca81-7d76-446b-b0fb-d03faceec4d5","Type":"ContainerStarted","Data":"e15edee2938ae732dd001b53804e283f32388f048dc1a008b4962482da939bb5"} Jul 24 22:53:43.920208 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:43.920180 2554 generic.go:356] "Generic (PLEG): container finished" podID="18e0ca81-7d76-446b-b0fb-d03faceec4d5" containerID="729e9242ad070f11047658759a3bb3ea7bd134262b63a4e7606ae233063f0f70" exitCode=0 Jul 24 22:53:43.920571 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:43.920254 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" event={"ID":"18e0ca81-7d76-446b-b0fb-d03faceec4d5","Type":"ContainerDied","Data":"729e9242ad070f11047658759a3bb3ea7bd134262b63a4e7606ae233063f0f70"} Jul 24 22:53:45.068081 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.068059 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:45.142625 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.142597 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e0ca81-7d76-446b-b0fb-d03faceec4d5-maas-api-service-ca\") pod \"18e0ca81-7d76-446b-b0fb-d03faceec4d5\" (UID: \"18e0ca81-7d76-446b-b0fb-d03faceec4d5\") " Jul 24 22:53:45.142923 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.142902 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e0ca81-7d76-446b-b0fb-d03faceec4d5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18e0ca81-7d76-446b-b0fb-d03faceec4d5" (UID: "18e0ca81-7d76-446b-b0fb-d03faceec4d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:53:45.243914 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.243886 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18e0ca81-7d76-446b-b0fb-d03faceec4d5-maas-api-service-ca\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:53:45.935822 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.935794 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" Jul 24 22:53:45.936028 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.935841 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk" event={"ID":"18e0ca81-7d76-446b-b0fb-d03faceec4d5","Type":"ContainerDied","Data":"e15edee2938ae732dd001b53804e283f32388f048dc1a008b4962482da939bb5"} Jul 24 22:53:45.936028 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:45.935922 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e15edee2938ae732dd001b53804e283f32388f048dc1a008b4962482da939bb5" Jul 24 22:53:52.828461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.828424 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645"] Jul 24 22:53:52.828935 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.828680 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" podUID="62e21225-27d7-4776-8736-c0bb300956f7" containerName="payload-processing" containerID="cri-o://ab0aca56205a3a99715264676346dc41bfa50ba1aaef1b55c0c5f83e8b7346ee" gracePeriod=30 Jul 24 22:53:52.834358 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.833964 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp"] Jul 24 22:53:52.834358 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.834190 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" podUID="9caea192-ffe7-466c-8c2e-496ced3fc700" containerName="payload-pre-processing" containerID="cri-o://7a64be530c4c4bb6d444184b097261982c05ec23c0a87cabec432c6a9edcad88" gracePeriod=30 Jul 24 22:53:52.971139 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.971094 2554 generic.go:356] "Generic (PLEG): container finished" podID="62e21225-27d7-4776-8736-c0bb300956f7" containerID="ab0aca56205a3a99715264676346dc41bfa50ba1aaef1b55c0c5f83e8b7346ee" exitCode=0 Jul 24 22:53:52.971338 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.971179 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" event={"ID":"62e21225-27d7-4776-8736-c0bb300956f7","Type":"ContainerDied","Data":"ab0aca56205a3a99715264676346dc41bfa50ba1aaef1b55c0c5f83e8b7346ee"} Jul 24 22:53:52.973042 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.972906 2554 generic.go:356] "Generic (PLEG): container finished" podID="9caea192-ffe7-466c-8c2e-496ced3fc700" containerID="7a64be530c4c4bb6d444184b097261982c05ec23c0a87cabec432c6a9edcad88" exitCode=0 Jul 24 22:53:52.973042 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:52.972977 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" event={"ID":"9caea192-ffe7-466c-8c2e-496ced3fc700","Type":"ContainerDied","Data":"7a64be530c4c4bb6d444184b097261982c05ec23c0a87cabec432c6a9edcad88"} Jul 24 22:53:53.119558 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.119532 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:53:53.122088 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.122069 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:53:53.199343 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.199316 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwk29\" (UniqueName: \"kubernetes.io/projected/9caea192-ffe7-466c-8c2e-496ced3fc700-kube-api-access-hwk29\") pod \"9caea192-ffe7-466c-8c2e-496ced3fc700\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " Jul 24 22:53:53.199482 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.199372 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9caea192-ffe7-466c-8c2e-496ced3fc700-plugins-config-volume\") pod \"9caea192-ffe7-466c-8c2e-496ced3fc700\" (UID: \"9caea192-ffe7-466c-8c2e-496ced3fc700\") " Jul 24 22:53:53.199482 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.199398 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62e21225-27d7-4776-8736-c0bb300956f7-plugins-config-volume\") pod \"62e21225-27d7-4776-8736-c0bb300956f7\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " Jul 24 22:53:53.199482 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.199448 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24mjd\" (UniqueName: \"kubernetes.io/projected/62e21225-27d7-4776-8736-c0bb300956f7-kube-api-access-24mjd\") pod \"62e21225-27d7-4776-8736-c0bb300956f7\" (UID: \"62e21225-27d7-4776-8736-c0bb300956f7\") " Jul 24 22:53:53.199734 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.199713 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9caea192-ffe7-466c-8c2e-496ced3fc700-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9caea192-ffe7-466c-8c2e-496ced3fc700" (UID: "9caea192-ffe7-466c-8c2e-496ced3fc700"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:53:53.199820 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.199728 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62e21225-27d7-4776-8736-c0bb300956f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "62e21225-27d7-4776-8736-c0bb300956f7" (UID: "62e21225-27d7-4776-8736-c0bb300956f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:53:53.201492 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.201471 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9caea192-ffe7-466c-8c2e-496ced3fc700-kube-api-access-hwk29" (OuterVolumeSpecName: "kube-api-access-hwk29") pod "9caea192-ffe7-466c-8c2e-496ced3fc700" (UID: "9caea192-ffe7-466c-8c2e-496ced3fc700"). InnerVolumeSpecName "kube-api-access-hwk29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:53:53.201606 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.201585 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62e21225-27d7-4776-8736-c0bb300956f7-kube-api-access-24mjd" (OuterVolumeSpecName: "kube-api-access-24mjd") pod "62e21225-27d7-4776-8736-c0bb300956f7" (UID: "62e21225-27d7-4776-8736-c0bb300956f7"). InnerVolumeSpecName "kube-api-access-24mjd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:53:53.300386 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.300362 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9caea192-ffe7-466c-8c2e-496ced3fc700-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:53:53.300386 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.300382 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62e21225-27d7-4776-8736-c0bb300956f7-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:53:53.300509 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.300390 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24mjd\" (UniqueName: \"kubernetes.io/projected/62e21225-27d7-4776-8736-c0bb300956f7-kube-api-access-24mjd\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:53:53.300509 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.300399 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwk29\" (UniqueName: \"kubernetes.io/projected/9caea192-ffe7-466c-8c2e-496ced3fc700-kube-api-access-hwk29\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:53:53.978917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.978893 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" Jul 24 22:53:53.979326 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.978942 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp" event={"ID":"9caea192-ffe7-466c-8c2e-496ced3fc700","Type":"ContainerDied","Data":"0268d5a2380bb979e703753e795c88326d85b15234afb0a0ae2a31664abec5ae"} Jul 24 22:53:53.979326 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.979019 2554 scope.go:117] "RemoveContainer" containerID="7a64be530c4c4bb6d444184b097261982c05ec23c0a87cabec432c6a9edcad88" Jul 24 22:53:53.980373 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.980219 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" event={"ID":"62e21225-27d7-4776-8736-c0bb300956f7","Type":"ContainerDied","Data":"518a1076aef9f8f95f036d3faa9cb10e63051b59059fb1394eb143c3c74f9796"} Jul 24 22:53:53.980373 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.980238 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645" Jul 24 22:53:53.988696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:53.988679 2554 scope.go:117] "RemoveContainer" containerID="ab0aca56205a3a99715264676346dc41bfa50ba1aaef1b55c0c5f83e8b7346ee" Jul 24 22:53:54.003021 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:54.002999 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp"] Jul 24 22:53:54.004931 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:54.004910 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-11557e-7ccbb4bf6-cmssp"] Jul 24 22:53:54.013832 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:54.013810 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645"] Jul 24 22:53:54.017637 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:54.017619 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-11557e-5bc756d49d-jf645"] Jul 24 22:53:55.174624 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:55.174596 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62e21225-27d7-4776-8736-c0bb300956f7" path="/var/lib/kubelet/pods/62e21225-27d7-4776-8736-c0bb300956f7/volumes" Jul 24 22:53:55.174956 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:55.174932 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9caea192-ffe7-466c-8c2e-496ced3fc700" path="/var/lib/kubelet/pods/9caea192-ffe7-466c-8c2e-496ced3fc700/volumes" Jul 24 22:53:57.595943 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:57.595911 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk"] Jul 24 22:53:57.598359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:57.598340 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11557e-j2bnk"] Jul 24 22:53:59.175620 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:53:59.175546 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e0ca81-7d76-446b-b0fb-d03faceec4d5" path="/var/lib/kubelet/pods/18e0ca81-7d76-446b-b0fb-d03faceec4d5/volumes" Jul 24 22:54:03.894880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:03.894847 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg"] Jul 24 22:54:03.895344 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:03.895108 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" containerID="cri-o://952347e31603275b4cdbb319945894acb3a57f9cfd5fb956541691d040b9d827" gracePeriod=30 Jul 24 22:54:03.956784 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:03.956754 2554 patch_prober.go:28] interesting pod/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 22:54:03.956932 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:03.956805 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 22:54:09.057258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.057227 2554 generic.go:356] "Generic (PLEG): container finished" podID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerID="952347e31603275b4cdbb319945894acb3a57f9cfd5fb956541691d040b9d827" exitCode=0 Jul 24 22:54:09.057713 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.057293 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" event={"ID":"6e9c8ab8-1eca-4563-a699-c04b09e4a379","Type":"ContainerDied","Data":"952347e31603275b4cdbb319945894acb3a57f9cfd5fb956541691d040b9d827"} Jul 24 22:54:09.057713 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.057358 2554 scope.go:117] "RemoveContainer" containerID="aaed2fc940b43d0ef74b172724b9996195d19a38bace6c8e15eeec5564e9f49b" Jul 24 22:54:09.147367 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.147341 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:54:09.316932 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.316857 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-data\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.316932 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.316894 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-socket\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.316943 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj5jw\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-kube-api-access-pj5jw\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.316986 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-token\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317029 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-podinfo\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317137 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317070 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-envoy\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317209 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istiod-ca-cert\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317230 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:09.317393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317263 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-credential-socket\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317291 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-certs\") pod \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\" (UID: \"6e9c8ab8-1eca-4563-a699-c04b09e4a379\") " Jul 24 22:54:09.317603 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317543 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:54:09.317603 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317569 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.317717 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317641 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:09.318084 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317963 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:09.318084 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.317986 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:09.319397 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.319375 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:54:09.319469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.319451 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-kube-api-access-pj5jw" (OuterVolumeSpecName: "kube-api-access-pj5jw") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "kube-api-access-pj5jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:54:09.319469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.319451 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:54:09.319576 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.319537 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e9c8ab8-1eca-4563-a699-c04b09e4a379" (UID: "6e9c8ab8-1eca-4563-a699-c04b09e4a379"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:09.418183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418130 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418181 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418183 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418190 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418198 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418206 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj5jw\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-kube-api-access-pj5jw\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418213 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418221 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:09.418361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:09.418228 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9c8ab8-1eca-4563-a699-c04b09e4a379-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:10.064742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:10.064714 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" Jul 24 22:54:10.064742 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:10.064725 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg" event={"ID":"6e9c8ab8-1eca-4563-a699-c04b09e4a379","Type":"ContainerDied","Data":"308989a18627de3636f8919c22eb982a27d051c8b799235484dafb9e3d384399"} Jul 24 22:54:10.065234 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:10.064776 2554 scope.go:117] "RemoveContainer" containerID="952347e31603275b4cdbb319945894acb3a57f9cfd5fb956541691d040b9d827" Jul 24 22:54:10.087132 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:10.087107 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg"] Jul 24 22:54:10.091158 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:10.091130 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-11557e-openshift-default-6665b67986-ftkbg"] Jul 24 22:54:11.175363 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:11.175333 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" path="/var/lib/kubelet/pods/6e9c8ab8-1eca-4563-a699-c04b09e4a379/volumes" Jul 24 22:54:14.664000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:14.663968 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz"] Jul 24 22:54:14.664858 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:14.664232 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" podUID="5b915f38-68d1-41d6-a026-3dfacd317c92" containerName="payload-processing" containerID="cri-o://a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38" gracePeriod=30 Jul 24 22:54:14.670745 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:14.670715 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl"] Jul 24 22:54:14.670997 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:14.670942 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" podUID="351d7d76-8482-43ab-9f8e-a4521be8003f" containerName="payload-pre-processing" containerID="cri-o://4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603" gracePeriod=30 Jul 24 22:54:14.911075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:14.911051 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:54:14.941877 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:14.941853 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:54:15.060141 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.060111 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txbpd\" (UniqueName: \"kubernetes.io/projected/351d7d76-8482-43ab-9f8e-a4521be8003f-kube-api-access-txbpd\") pod \"351d7d76-8482-43ab-9f8e-a4521be8003f\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " Jul 24 22:54:15.060285 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.060204 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g6cq\" (UniqueName: \"kubernetes.io/projected/5b915f38-68d1-41d6-a026-3dfacd317c92-kube-api-access-4g6cq\") pod \"5b915f38-68d1-41d6-a026-3dfacd317c92\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " Jul 24 22:54:15.060285 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.060249 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351d7d76-8482-43ab-9f8e-a4521be8003f-plugins-config-volume\") pod \"351d7d76-8482-43ab-9f8e-a4521be8003f\" (UID: \"351d7d76-8482-43ab-9f8e-a4521be8003f\") " Jul 24 22:54:15.060357 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.060322 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b915f38-68d1-41d6-a026-3dfacd317c92-plugins-config-volume\") pod \"5b915f38-68d1-41d6-a026-3dfacd317c92\" (UID: \"5b915f38-68d1-41d6-a026-3dfacd317c92\") " Jul 24 22:54:15.060718 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.060695 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b915f38-68d1-41d6-a026-3dfacd317c92-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b915f38-68d1-41d6-a026-3dfacd317c92" (UID: "5b915f38-68d1-41d6-a026-3dfacd317c92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:54:15.060969 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.060692 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351d7d76-8482-43ab-9f8e-a4521be8003f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "351d7d76-8482-43ab-9f8e-a4521be8003f" (UID: "351d7d76-8482-43ab-9f8e-a4521be8003f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:54:15.062306 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.062285 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351d7d76-8482-43ab-9f8e-a4521be8003f-kube-api-access-txbpd" (OuterVolumeSpecName: "kube-api-access-txbpd") pod "351d7d76-8482-43ab-9f8e-a4521be8003f" (UID: "351d7d76-8482-43ab-9f8e-a4521be8003f"). InnerVolumeSpecName "kube-api-access-txbpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:54:15.062390 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.062358 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b915f38-68d1-41d6-a026-3dfacd317c92-kube-api-access-4g6cq" (OuterVolumeSpecName: "kube-api-access-4g6cq") pod "5b915f38-68d1-41d6-a026-3dfacd317c92" (UID: "5b915f38-68d1-41d6-a026-3dfacd317c92"). InnerVolumeSpecName "kube-api-access-4g6cq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:54:15.090497 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.090468 2554 generic.go:356] "Generic (PLEG): container finished" podID="351d7d76-8482-43ab-9f8e-a4521be8003f" containerID="4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603" exitCode=0 Jul 24 22:54:15.090593 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.090507 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" Jul 24 22:54:15.090593 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.090549 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" event={"ID":"351d7d76-8482-43ab-9f8e-a4521be8003f","Type":"ContainerDied","Data":"4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603"} Jul 24 22:54:15.090693 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.090626 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl" event={"ID":"351d7d76-8482-43ab-9f8e-a4521be8003f","Type":"ContainerDied","Data":"e06efddc79b9701375112df5125a4f3ec7062c6bc6e98061aeb4654e86aad930"} Jul 24 22:54:15.090693 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.090653 2554 scope.go:117] "RemoveContainer" containerID="4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603" Jul 24 22:54:15.091868 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.091842 2554 generic.go:356] "Generic (PLEG): container finished" podID="5b915f38-68d1-41d6-a026-3dfacd317c92" containerID="a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38" exitCode=0 Jul 24 22:54:15.091944 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.091882 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" Jul 24 22:54:15.091944 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.091893 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" event={"ID":"5b915f38-68d1-41d6-a026-3dfacd317c92","Type":"ContainerDied","Data":"a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38"} Jul 24 22:54:15.091944 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.091925 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz" event={"ID":"5b915f38-68d1-41d6-a026-3dfacd317c92","Type":"ContainerDied","Data":"8783b52b04fea2c035f2ed553530031a547bc2cef822d57fe9b814d7bc1ca5ab"} Jul 24 22:54:15.099370 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.099351 2554 scope.go:117] "RemoveContainer" containerID="4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603" Jul 24 22:54:15.099647 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:54:15.099624 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603\": container with ID starting with 4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603 not found: ID does not exist" containerID="4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603" Jul 24 22:54:15.099736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.099659 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603"} err="failed to get container status \"4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603\": rpc error: code = NotFound desc = could not find container \"4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603\": container with ID starting with 4509534e066bb0c33e9d8af447eb1af383d0591be7cbd71250d1527ecbbb5603 not found: ID does not exist" Jul 24 22:54:15.099736 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.099682 2554 scope.go:117] "RemoveContainer" containerID="a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38" Jul 24 22:54:15.107443 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.107425 2554 scope.go:117] "RemoveContainer" containerID="a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38" Jul 24 22:54:15.107669 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:54:15.107651 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38\": container with ID starting with a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38 not found: ID does not exist" containerID="a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38" Jul 24 22:54:15.107720 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.107677 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38"} err="failed to get container status \"a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38\": rpc error: code = NotFound desc = could not find container \"a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38\": container with ID starting with a66debf497886fad4a74647c07009e81069a096415e207301835d522a97bfa38 not found: ID does not exist" Jul 24 22:54:15.116380 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.116359 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl"] Jul 24 22:54:15.120788 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.120763 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-422c4e-7c9f7c586d-ztrsl"] Jul 24 22:54:15.130935 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.130917 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz"] Jul 24 22:54:15.133717 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.133698 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-422c4e-78454df584-4nxsz"] Jul 24 22:54:15.161640 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.161618 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/351d7d76-8482-43ab-9f8e-a4521be8003f-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:15.161640 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.161636 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b915f38-68d1-41d6-a026-3dfacd317c92-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:15.161763 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.161646 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txbpd\" (UniqueName: \"kubernetes.io/projected/351d7d76-8482-43ab-9f8e-a4521be8003f-kube-api-access-txbpd\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:15.161763 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.161655 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g6cq\" (UniqueName: \"kubernetes.io/projected/5b915f38-68d1-41d6-a026-3dfacd317c92-kube-api-access-4g6cq\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:15.176160 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.176094 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351d7d76-8482-43ab-9f8e-a4521be8003f" path="/var/lib/kubelet/pods/351d7d76-8482-43ab-9f8e-a4521be8003f/volumes" Jul 24 22:54:15.176456 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:15.176442 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b915f38-68d1-41d6-a026-3dfacd317c92" path="/var/lib/kubelet/pods/5b915f38-68d1-41d6-a026-3dfacd317c92/volumes" Jul 24 22:54:26.051055 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.051023 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266"] Jul 24 22:54:26.051554 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.051230 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" containerID="cri-o://981bca4da5e4efb20ec61610ce031774b45927a549db4610ca768ece59b50f71" gracePeriod=30 Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.933792 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz"] Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934421 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62e21225-27d7-4776-8736-c0bb300956f7" containerName="payload-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934442 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="62e21225-27d7-4776-8736-c0bb300956f7" containerName="payload-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934469 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9caea192-ffe7-466c-8c2e-496ced3fc700" containerName="payload-pre-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934477 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9caea192-ffe7-466c-8c2e-496ced3fc700" containerName="payload-pre-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934487 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934495 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934520 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b915f38-68d1-41d6-a026-3dfacd317c92" containerName="payload-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934527 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b915f38-68d1-41d6-a026-3dfacd317c92" containerName="payload-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934541 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934555 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934566 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e0ca81-7d76-446b-b0fb-d03faceec4d5" containerName="revoke-keys" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934574 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e0ca81-7d76-446b-b0fb-d03faceec4d5" containerName="revoke-keys" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934603 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351d7d76-8482-43ab-9f8e-a4521be8003f" containerName="payload-pre-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934611 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="351d7d76-8482-43ab-9f8e-a4521be8003f" containerName="payload-pre-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934740 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934751 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b915f38-68d1-41d6-a026-3dfacd317c92" containerName="payload-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934761 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e0ca81-7d76-446b-b0fb-d03faceec4d5" containerName="revoke-keys" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934776 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="62e21225-27d7-4776-8736-c0bb300956f7" containerName="payload-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934786 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e9c8ab8-1eca-4563-a699-c04b09e4a379" containerName="istio-proxy" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934804 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9caea192-ffe7-466c-8c2e-496ced3fc700" containerName="payload-pre-processing" Jul 24 22:54:26.935361 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.934814 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="351d7d76-8482-43ab-9f8e-a4521be8003f" containerName="payload-pre-processing" Jul 24 22:54:26.939653 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.939636 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.942900 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.942865 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-aac4b8-openshift-default-dockercfg-qjn4h\"" Jul 24 22:54:26.950328 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.950306 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz"] Jul 24 22:54:26.955809 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.955783 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-socket\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.955922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.955847 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-podinfo\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.955922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.955885 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-envoy\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.955922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.955913 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-data\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.956100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.955987 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-credential-socket\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.956100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.956051 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istiod-ca-cert\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.956100 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.956093 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-certs\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.956259 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.956119 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-token\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:26.956259 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:26.956168 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkvs9\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-kube-api-access-rkvs9\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.056569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056535 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-envoy\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.056569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056572 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-data\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056594 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-credential-socket\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056620 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istiod-ca-cert\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056643 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-certs\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056668 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-token\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056719 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkvs9\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-kube-api-access-rkvs9\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056763 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-socket\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057054 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.056827 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-podinfo\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.057061 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-data\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.057069 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-credential-socket\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.057200 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-certs\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057448 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.057337 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-socket\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.057644 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.057486 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istiod-ca-cert\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.059728 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.059697 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-envoy\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.059839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.059805 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-podinfo\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.065631 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.065609 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-token\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.066339 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.066315 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkvs9\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-kube-api-access-rkvs9\") pod \"e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.251277 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.251182 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:27.386766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.386741 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz"] Jul 24 22:54:27.388590 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:54:27.388563 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab68bc2a_c3d4_421c_9b8e_2a17a3186e7e.slice/crio-9880d528bf39c3c767293c5f1e69f78d3936356063bf0f733e0959e76b62c4e3 WatchSource:0}: Error finding container 9880d528bf39c3c767293c5f1e69f78d3936356063bf0f733e0959e76b62c4e3: Status 404 returned error can't find the container with id 9880d528bf39c3c767293c5f1e69f78d3936356063bf0f733e0959e76b62c4e3 Jul 24 22:54:27.390677 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.390642 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:27.390752 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.390708 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:27.390752 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:27.390744 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:28.157687 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:28.157659 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" event={"ID":"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e","Type":"ContainerStarted","Data":"6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633"} Jul 24 22:54:28.158036 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:28.157708 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" event={"ID":"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e","Type":"ContainerStarted","Data":"9880d528bf39c3c767293c5f1e69f78d3936356063bf0f733e0959e76b62c4e3"} Jul 24 22:54:28.181740 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:28.181690 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podStartSLOduration=2.181677366 podStartE2EDuration="2.181677366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:28.179966773 +0000 UTC m=+2365.566784820" watchObservedRunningTime="2026-07-24 22:54:28.181677366 +0000 UTC m=+2365.568495410" Jul 24 22:54:28.251679 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:28.251651 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:28.256124 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:28.256104 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:29.162652 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:29.162626 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:29.163514 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:29.163496 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:31.047955 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.047923 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc"] Jul 24 22:54:31.051509 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.051490 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.054812 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.054790 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-aac4b8-dockercfg-hrgtx\"" Jul 24 22:54:31.054925 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.054905 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-aac4b8\"" Jul 24 22:54:31.068142 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.068115 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc"] Jul 24 22:54:31.070140 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.070111 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f"] Jul 24 22:54:31.073394 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.073379 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.085787 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.085768 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f"] Jul 24 22:54:31.088678 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.088652 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vxvr\" (UniqueName: \"kubernetes.io/projected/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-kube-api-access-9vxvr\") pod \"payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.088764 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.088706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.175794 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.175762 2554 generic.go:356] "Generic (PLEG): container finished" podID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerID="981bca4da5e4efb20ec61610ce031774b45927a549db4610ca768ece59b50f71" exitCode=0 Jul 24 22:54:31.177473 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.177443 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" event={"ID":"393a24aa-2ccb-43f7-8856-9a0fd126ee01","Type":"ContainerDied","Data":"981bca4da5e4efb20ec61610ce031774b45927a549db4610ca768ece59b50f71"} Jul 24 22:54:31.177586 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.177515 2554 scope.go:117] "RemoveContainer" containerID="8f57462014a768813fe618412ba81ba27a2a47277a61aa26052b785bf82be778" Jul 24 22:54:31.189990 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.189967 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/a0677508-a525-4f15-b8d4-5681080df376-kube-api-access-q6x68\") pod \"payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.190112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.190018 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxvr\" (UniqueName: \"kubernetes.io/projected/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-kube-api-access-9vxvr\") pod \"payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.190112 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.190054 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0677508-a525-4f15-b8d4-5681080df376-plugins-config-volume\") pod \"payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.190469 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.190445 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.191220 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.191193 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.199057 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.199015 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vxvr\" (UniqueName: \"kubernetes.io/projected/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-kube-api-access-9vxvr\") pod \"payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.290973 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.290944 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0677508-a525-4f15-b8d4-5681080df376-plugins-config-volume\") pod \"payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.291102 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.291075 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/a0677508-a525-4f15-b8d4-5681080df376-kube-api-access-q6x68\") pod \"payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.291748 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.291727 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0677508-a525-4f15-b8d4-5681080df376-plugins-config-volume\") pod \"payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.295614 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.295597 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:54:31.299595 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.299546 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/a0677508-a525-4f15-b8d4-5681080df376-kube-api-access-q6x68\") pod \"payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.362918 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.362891 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:31.391909 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.391885 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-podinfo\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.391931 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-token\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.391948 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-certs\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.391979 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-envoy\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392001 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-data\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392029 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istiod-ca-cert\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392059 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-credential-socket\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392439 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392086 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-socket\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392439 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392162 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbjl5\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-kube-api-access-fbjl5\") pod \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\" (UID: \"393a24aa-2ccb-43f7-8856-9a0fd126ee01\") " Jul 24 22:54:31.392439 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392277 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:31.392656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392519 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:31.392656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392542 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.392921 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392862 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:54:31.393031 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.392969 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-data" (OuterVolumeSpecName: "istio-data") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:31.393140 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.393105 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:31.394360 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.394338 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-token" (OuterVolumeSpecName: "istio-token") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:54:31.394449 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.394357 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:54:31.394906 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.394888 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:54:31.394950 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.394905 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-kube-api-access-fbjl5" (OuterVolumeSpecName: "kube-api-access-fbjl5") pod "393a24aa-2ccb-43f7-8856-9a0fd126ee01" (UID: "393a24aa-2ccb-43f7-8856-9a0fd126ee01"). InnerVolumeSpecName "kube-api-access-fbjl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:54:31.425718 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.425692 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493698 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493727 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493739 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493750 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493761 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/393a24aa-2ccb-43f7-8856-9a0fd126ee01-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493772 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493808 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/393a24aa-2ccb-43f7-8856-9a0fd126ee01-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.493941 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.493820 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbjl5\" (UniqueName: \"kubernetes.io/projected/393a24aa-2ccb-43f7-8856-9a0fd126ee01-kube-api-access-fbjl5\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:31.497214 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.497136 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc"] Jul 24 22:54:31.498449 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:54:31.498426 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c81dbb_4e55_44c9_a8c7_55f022ef94f2.slice/crio-1ebfe0301a79c038cc5b665978971f054b54f2309de7f74277156c7f978661a2 WatchSource:0}: Error finding container 1ebfe0301a79c038cc5b665978971f054b54f2309de7f74277156c7f978661a2: Status 404 returned error can't find the container with id 1ebfe0301a79c038cc5b665978971f054b54f2309de7f74277156c7f978661a2 Jul 24 22:54:31.566880 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:31.566845 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f"] Jul 24 22:54:31.568123 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:54:31.568097 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0677508_a525_4f15_b8d4_5681080df376.slice/crio-058f4255391ea108266f397ab1aaa2a1e51b2ee65500b9c3c704134f9e7eb15a WatchSource:0}: Error finding container 058f4255391ea108266f397ab1aaa2a1e51b2ee65500b9c3c704134f9e7eb15a: Status 404 returned error can't find the container with id 058f4255391ea108266f397ab1aaa2a1e51b2ee65500b9c3c704134f9e7eb15a Jul 24 22:54:32.182837 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.182804 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" event={"ID":"a0677508-a525-4f15-b8d4-5681080df376","Type":"ContainerStarted","Data":"c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2"} Jul 24 22:54:32.183237 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.182861 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" event={"ID":"a0677508-a525-4f15-b8d4-5681080df376","Type":"ContainerStarted","Data":"058f4255391ea108266f397ab1aaa2a1e51b2ee65500b9c3c704134f9e7eb15a"} Jul 24 22:54:32.183237 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.182902 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:32.184189 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.184144 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" Jul 24 22:54:32.184189 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.184169 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266" event={"ID":"393a24aa-2ccb-43f7-8856-9a0fd126ee01","Type":"ContainerDied","Data":"49b733be65f943b7b04cd6a549a21109254e504af14f8864b6d90790eec24c28"} Jul 24 22:54:32.184389 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.184225 2554 scope.go:117] "RemoveContainer" containerID="981bca4da5e4efb20ec61610ce031774b45927a549db4610ca768ece59b50f71" Jul 24 22:54:32.185664 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.185637 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" event={"ID":"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2","Type":"ContainerStarted","Data":"8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660"} Jul 24 22:54:32.185749 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.185682 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" event={"ID":"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2","Type":"ContainerStarted","Data":"1ebfe0301a79c038cc5b665978971f054b54f2309de7f74277156c7f978661a2"} Jul 24 22:54:32.185749 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.185702 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:32.205496 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.205455 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" podStartSLOduration=1.205442518 podStartE2EDuration="1.205442518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:32.202816042 +0000 UTC m=+2369.589634090" watchObservedRunningTime="2026-07-24 22:54:32.205442518 +0000 UTC m=+2369.592260565" Jul 24 22:54:32.219491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.219454 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" podStartSLOduration=1.219444989 podStartE2EDuration="1.219444989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:32.218899084 +0000 UTC m=+2369.605717133" watchObservedRunningTime="2026-07-24 22:54:32.219444989 +0000 UTC m=+2369.606263035" Jul 24 22:54:32.233502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.233482 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266"] Jul 24 22:54:32.238018 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:32.237997 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-422c4e-openshift-default-7585b75796-5x266"] Jul 24 22:54:33.175510 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:33.175472 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" path="/var/lib/kubelet/pods/393a24aa-2ccb-43f7-8856-9a0fd126ee01/volumes" Jul 24 22:54:35.308778 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.308746 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f"] Jul 24 22:54:35.309276 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.309140 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" Jul 24 22:54:35.309276 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.309175 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" Jul 24 22:54:35.309276 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.309203 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" Jul 24 22:54:35.309276 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.309211 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" Jul 24 22:54:35.309461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.309298 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" Jul 24 22:54:35.309461 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.309312 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="393a24aa-2ccb-43f7-8856-9a0fd126ee01" containerName="istio-proxy" Jul 24 22:54:35.313656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.313638 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.316442 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.316422 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-76c851-openshift-default-dockercfg-mstnq\"" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323301 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-certs\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323336 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-envoy\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323374 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-credential-socket\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323418 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-istio-token\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323452 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2182204-442b-4a2f-b062-d1349d328748-istiod-ca-cert\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323477 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-socket\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323497 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2182204-442b-4a2f-b062-d1349d328748-istio-podinfo\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323578 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-data\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.323723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.323648 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7575\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-kube-api-access-t7575\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.326211 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.326187 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f"] Jul 24 22:54:35.424629 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424588 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-envoy\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.424803 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424637 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-credential-socket\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.424803 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424667 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-istio-token\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.424803 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424691 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2182204-442b-4a2f-b062-d1349d328748-istiod-ca-cert\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.424803 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424718 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-socket\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.424803 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424736 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2182204-442b-4a2f-b062-d1349d328748-istio-podinfo\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.424803 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424770 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-data\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424849 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7575\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-kube-api-access-t7575\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.424891 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-certs\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.425132 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-credential-socket\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.425220 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-certs\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425332 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.425272 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-data\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425511 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.425490 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-socket\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.425556 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.425496 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2182204-442b-4a2f-b062-d1349d328748-istiod-ca-cert\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.427061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.427036 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-envoy\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.427433 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.427410 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2182204-442b-4a2f-b062-d1349d328748-istio-podinfo\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.433162 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.433116 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-istio-token\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.433355 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.433339 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7575\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-kube-api-access-t7575\") pod \"e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.624631 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.624543 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:35.763657 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.763623 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f"] Jul 24 22:54:35.764990 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:54:35.764965 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2182204_442b_4a2f_b062_d1349d328748.slice/crio-6e23c1ff983acd5eddf1089900d2ef3eb9fe3f1c3ec880026b35b4d339a785a5 WatchSource:0}: Error finding container 6e23c1ff983acd5eddf1089900d2ef3eb9fe3f1c3ec880026b35b4d339a785a5: Status 404 returned error can't find the container with id 6e23c1ff983acd5eddf1089900d2ef3eb9fe3f1c3ec880026b35b4d339a785a5 Jul 24 22:54:35.767215 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.767181 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:35.767318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.767263 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:35.767318 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:35.767309 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:36.209713 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:36.209676 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" event={"ID":"a2182204-442b-4a2f-b062-d1349d328748","Type":"ContainerStarted","Data":"fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e"} Jul 24 22:54:36.209908 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:36.209730 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" event={"ID":"a2182204-442b-4a2f-b062-d1349d328748","Type":"ContainerStarted","Data":"6e23c1ff983acd5eddf1089900d2ef3eb9fe3f1c3ec880026b35b4d339a785a5"} Jul 24 22:54:36.228204 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:36.228138 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podStartSLOduration=1.228118078 podStartE2EDuration="1.228118078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:36.227568509 +0000 UTC m=+2373.614386566" watchObservedRunningTime="2026-07-24 22:54:36.228118078 +0000 UTC m=+2373.614936125" Jul 24 22:54:36.625207 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:36.625124 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:36.629870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:36.629837 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:37.213988 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:37.213957 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:37.214950 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:37.214934 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:39.023873 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.023835 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p"] Jul 24 22:54:39.027185 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.027143 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.030550 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.030526 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-76c851\"" Jul 24 22:54:39.030683 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.030666 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-76c851-dockercfg-85cl2\"" Jul 24 22:54:39.037507 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.037485 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p"] Jul 24 22:54:39.059518 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.059472 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1272087a-c75d-4bbc-8f29-37d67244063e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.059726 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.059556 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsjts\" (UniqueName: \"kubernetes.io/projected/1272087a-c75d-4bbc-8f29-37d67244063e-kube-api-access-hsjts\") pod \"payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.115694 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.115656 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl"] Jul 24 22:54:39.118846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.118824 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.131296 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.131272 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl"] Jul 24 22:54:39.163601 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.163557 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-plugins-config-volume\") pod \"payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.163920 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.163899 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1272087a-c75d-4bbc-8f29-37d67244063e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.164080 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.164065 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsjts\" (UniqueName: \"kubernetes.io/projected/1272087a-c75d-4bbc-8f29-37d67244063e-kube-api-access-hsjts\") pod \"payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.164258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.164239 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6w2d\" (UniqueName: \"kubernetes.io/projected/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-kube-api-access-w6w2d\") pod \"payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.165298 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.165276 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1272087a-c75d-4bbc-8f29-37d67244063e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.183274 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.183241 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsjts\" (UniqueName: \"kubernetes.io/projected/1272087a-c75d-4bbc-8f29-37d67244063e-kube-api-access-hsjts\") pod \"payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.269179 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.268645 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6w2d\" (UniqueName: \"kubernetes.io/projected/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-kube-api-access-w6w2d\") pod \"payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.269179 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.268722 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-plugins-config-volume\") pod \"payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.273175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.270812 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-plugins-config-volume\") pod \"payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.285077 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.285002 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6w2d\" (UniqueName: \"kubernetes.io/projected/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-kube-api-access-w6w2d\") pod \"payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.339565 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.339531 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:39.429073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.429047 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:39.532094 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.532070 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p"] Jul 24 22:54:39.534505 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:54:39.534451 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1272087a_c75d_4bbc_8f29_37d67244063e.slice/crio-85ee2c56f0aa4ad98650317758b8c3e82c40f6b3f971640fbc66c33d1fb7d159 WatchSource:0}: Error finding container 85ee2c56f0aa4ad98650317758b8c3e82c40f6b3f971640fbc66c33d1fb7d159: Status 404 returned error can't find the container with id 85ee2c56f0aa4ad98650317758b8c3e82c40f6b3f971640fbc66c33d1fb7d159 Jul 24 22:54:39.635963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:39.635941 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl"] Jul 24 22:54:39.637500 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:54:39.637477 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b6b6e1_6177_4ad9_a9ee_629ea9d87091.slice/crio-c003fcff8a8dfcd0db636fc9247ab5901b0d3c5b1599b19d4973e72a8a10b4b1 WatchSource:0}: Error finding container c003fcff8a8dfcd0db636fc9247ab5901b0d3c5b1599b19d4973e72a8a10b4b1: Status 404 returned error can't find the container with id c003fcff8a8dfcd0db636fc9247ab5901b0d3c5b1599b19d4973e72a8a10b4b1 Jul 24 22:54:40.232777 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.232740 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" event={"ID":"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091","Type":"ContainerStarted","Data":"6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5"} Jul 24 22:54:40.233337 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.232798 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" event={"ID":"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091","Type":"ContainerStarted","Data":"c003fcff8a8dfcd0db636fc9247ab5901b0d3c5b1599b19d4973e72a8a10b4b1"} Jul 24 22:54:40.233485 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.233392 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:40.241432 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.241407 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" event={"ID":"1272087a-c75d-4bbc-8f29-37d67244063e","Type":"ContainerStarted","Data":"1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed"} Jul 24 22:54:40.241575 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.241471 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" event={"ID":"1272087a-c75d-4bbc-8f29-37d67244063e","Type":"ContainerStarted","Data":"85ee2c56f0aa4ad98650317758b8c3e82c40f6b3f971640fbc66c33d1fb7d159"} Jul 24 22:54:40.241575 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.241493 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:40.267267 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.267196 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" podStartSLOduration=1.258357684 podStartE2EDuration="1.258357684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:40.255286696 +0000 UTC m=+2377.642104745" watchObservedRunningTime="2026-07-24 22:54:40.258357684 +0000 UTC m=+2377.645175730" Jul 24 22:54:40.276275 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:40.276217 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" podStartSLOduration=1.276198651 podStartE2EDuration="1.276198651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:54:40.275668065 +0000 UTC m=+2377.662486113" watchObservedRunningTime="2026-07-24 22:54:40.276198651 +0000 UTC m=+2377.663016701" Jul 24 22:54:43.195025 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:43.194935 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:54:43.195025 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:43.194992 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:54:45.206312 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:45.206273 2554 patch_prober.go:28] interesting pod/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 22:54:45.206763 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:45.206329 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 22:54:46.283658 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:46.283627 2554 generic.go:356] "Generic (PLEG): container finished" podID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerID="6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633" exitCode=0 Jul 24 22:54:46.284107 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:46.283706 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" event={"ID":"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e","Type":"ContainerDied","Data":"6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633"} Jul 24 22:54:46.284107 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:46.284088 2554 scope.go:117] "RemoveContainer" containerID="6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633" Jul 24 22:54:46.285209 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:46.285184 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:46.285266 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:46.285246 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:46.285306 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:46.285279 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:47.291786 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:47.291747 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" event={"ID":"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e","Type":"ContainerStarted","Data":"9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728"} Jul 24 22:54:48.252485 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:48.252440 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:48.252706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:48.252582 2554 patch_prober.go:28] interesting pod/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 22:54:48.252706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:48.252627 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 22:54:49.252462 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:49.252417 2554 patch_prober.go:28] interesting pod/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 22:54:49.252971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:49.252488 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 22:54:50.251839 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.251788 2554 patch_prober.go:28] interesting pod/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 22:54:50.252074 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.251868 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 22:54:50.467798 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.467734 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll"] Jul 24 22:54:50.472368 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.472286 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:50.475340 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.475102 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 22:54:50.478520 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.476734 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll"] Jul 24 22:54:50.479668 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.479445 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcbdx\"" Jul 24 22:54:50.580730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.580606 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d2dece-dfc8-467d-adeb-3d4184a8461f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-aac4b8-smsll\" (UID: \"55d2dece-dfc8-467d-adeb-3d4184a8461f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:50.682458 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.681484 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d2dece-dfc8-467d-adeb-3d4184a8461f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-aac4b8-smsll\" (UID: \"55d2dece-dfc8-467d-adeb-3d4184a8461f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:50.682458 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.682402 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d2dece-dfc8-467d-adeb-3d4184a8461f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-aac4b8-smsll\" (UID: \"55d2dece-dfc8-467d-adeb-3d4184a8461f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:50.795862 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:50.795790 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:51.274801 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:51.271766 2554 patch_prober.go:28] interesting pod/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 22:54:51.274801 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:51.271852 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 22:54:51.279713 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:51.278321 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:54:51.280730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:51.280422 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:54:51.283347 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:51.282727 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll"] Jul 24 22:54:51.322058 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:51.322019 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" event={"ID":"55d2dece-dfc8-467d-adeb-3d4184a8461f","Type":"ContainerStarted","Data":"539e61b70f6ef52ea8687b2356405f3a8a19be269645896551d7b418e11b4626"} Jul 24 22:54:52.255244 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:52.255215 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:52.255680 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:52.255616 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:52.256333 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:52.256313 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:54:52.331709 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:52.331673 2554 generic.go:356] "Generic (PLEG): container finished" podID="55d2dece-dfc8-467d-adeb-3d4184a8461f" containerID="d99a7b77ea0afd1839ae0acf60d1a3b5df814664257e3f4c6441cfcab97dd9ba" exitCode=0 Jul 24 22:54:52.331960 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:52.331763 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" event={"ID":"55d2dece-dfc8-467d-adeb-3d4184a8461f","Type":"ContainerDied","Data":"d99a7b77ea0afd1839ae0acf60d1a3b5df814664257e3f4c6441cfcab97dd9ba"} Jul 24 22:54:53.256310 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:53.256276 2554 patch_prober.go:28] interesting pod/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 22:54:53.256710 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:53.256321 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 22:54:53.474097 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:53.474073 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:53.612491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:53.612419 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d2dece-dfc8-467d-adeb-3d4184a8461f-maas-api-service-ca\") pod \"55d2dece-dfc8-467d-adeb-3d4184a8461f\" (UID: \"55d2dece-dfc8-467d-adeb-3d4184a8461f\") " Jul 24 22:54:53.612793 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:53.612773 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55d2dece-dfc8-467d-adeb-3d4184a8461f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55d2dece-dfc8-467d-adeb-3d4184a8461f" (UID: "55d2dece-dfc8-467d-adeb-3d4184a8461f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:54:53.714023 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:53.713989 2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55d2dece-dfc8-467d-adeb-3d4184a8461f-maas-api-service-ca\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:54:54.343609 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:54.343575 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" event={"ID":"55d2dece-dfc8-467d-adeb-3d4184a8461f","Type":"ContainerDied","Data":"539e61b70f6ef52ea8687b2356405f3a8a19be269645896551d7b418e11b4626"} Jul 24 22:54:54.343609 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:54.343597 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll" Jul 24 22:54:54.344036 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:54.343626 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="539e61b70f6ef52ea8687b2356405f3a8a19be269645896551d7b418e11b4626" Jul 24 22:54:55.357564 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:55.357527 2554 generic.go:356] "Generic (PLEG): container finished" podID="a2182204-442b-4a2f-b062-d1349d328748" containerID="fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e" exitCode=0 Jul 24 22:54:55.358000 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:55.357606 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" event={"ID":"a2182204-442b-4a2f-b062-d1349d328748","Type":"ContainerDied","Data":"fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e"} Jul 24 22:54:55.358046 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:55.357999 2554 scope.go:117] "RemoveContainer" containerID="fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e" Jul 24 22:54:55.359060 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:55.359029 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:55.359121 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:55.359106 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:55.359195 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:55.359176 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:54:56.366785 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:56.366754 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" event={"ID":"a2182204-442b-4a2f-b062-d1349d328748","Type":"ContainerStarted","Data":"e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d"} Jul 24 22:54:56.624770 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:56.624675 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:54:56.624972 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:56.624866 2554 patch_prober.go:28] interesting pod/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 22:54:56.624972 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:56.624919 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 22:54:57.625886 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:57.625854 2554 patch_prober.go:28] interesting pod/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 22:54:57.626324 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:57.625905 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 22:54:58.624865 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:58.624832 2554 patch_prober.go:28] interesting pod/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 22:54:58.625039 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:58.624884 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 22:54:59.625572 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:59.625534 2554 patch_prober.go:28] interesting pod/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 24 22:54:59.625959 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:54:59.625596 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 24 22:55:00.629922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:00.629890 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:55:00.630415 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:00.630174 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:55:00.630814 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:00.630798 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:55:05.712488 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:05.712450 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f"] Jul 24 22:55:05.712911 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:05.712727 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" podUID="a0677508-a525-4f15-b8d4-5681080df376" containerName="payload-processing" containerID="cri-o://c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2" gracePeriod=30 Jul 24 22:55:05.718955 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:05.718928 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc"] Jul 24 22:55:05.719142 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:05.719117 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" podUID="b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" containerName="payload-pre-processing" containerID="cri-o://8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660" gracePeriod=30 Jul 24 22:55:06.001463 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.001436 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:55:06.005925 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.005907 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:55:06.028670 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.028648 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0677508-a525-4f15-b8d4-5681080df376-plugins-config-volume\") pod \"a0677508-a525-4f15-b8d4-5681080df376\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " Jul 24 22:55:06.028785 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.028698 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/a0677508-a525-4f15-b8d4-5681080df376-kube-api-access-q6x68\") pod \"a0677508-a525-4f15-b8d4-5681080df376\" (UID: \"a0677508-a525-4f15-b8d4-5681080df376\") " Jul 24 22:55:06.028852 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.028818 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-plugins-config-volume\") pod \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " Jul 24 22:55:06.028912 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.028854 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vxvr\" (UniqueName: \"kubernetes.io/projected/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-kube-api-access-9vxvr\") pod \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\" (UID: \"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2\") " Jul 24 22:55:06.029143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.029113 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0677508-a525-4f15-b8d4-5681080df376-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0677508-a525-4f15-b8d4-5681080df376" (UID: "a0677508-a525-4f15-b8d4-5681080df376"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:55:06.029259 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.029170 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" (UID: "b4c81dbb-4e55-44c9-a8c7-55f022ef94f2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:55:06.030827 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.030808 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0677508-a525-4f15-b8d4-5681080df376-kube-api-access-q6x68" (OuterVolumeSpecName: "kube-api-access-q6x68") pod "a0677508-a525-4f15-b8d4-5681080df376" (UID: "a0677508-a525-4f15-b8d4-5681080df376"). InnerVolumeSpecName "kube-api-access-q6x68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:06.030972 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.030945 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-kube-api-access-9vxvr" (OuterVolumeSpecName: "kube-api-access-9vxvr") pod "b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" (UID: "b4c81dbb-4e55-44c9-a8c7-55f022ef94f2"). InnerVolumeSpecName "kube-api-access-9vxvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:06.129771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.129748 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0677508-a525-4f15-b8d4-5681080df376-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:06.129771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.129771 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6x68\" (UniqueName: \"kubernetes.io/projected/a0677508-a525-4f15-b8d4-5681080df376-kube-api-access-q6x68\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:06.129922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.129781 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:06.129922 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.129789 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vxvr\" (UniqueName: \"kubernetes.io/projected/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2-kube-api-access-9vxvr\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:06.424871 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.424846 2554 generic.go:356] "Generic (PLEG): container finished" podID="b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" containerID="8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660" exitCode=0 Jul 24 22:55:06.425051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.424886 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" Jul 24 22:55:06.425051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.424936 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" event={"ID":"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2","Type":"ContainerDied","Data":"8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660"} Jul 24 22:55:06.425051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.424987 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc" event={"ID":"b4c81dbb-4e55-44c9-a8c7-55f022ef94f2","Type":"ContainerDied","Data":"1ebfe0301a79c038cc5b665978971f054b54f2309de7f74277156c7f978661a2"} Jul 24 22:55:06.425051 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.425015 2554 scope.go:117] "RemoveContainer" containerID="8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660" Jul 24 22:55:06.426314 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.426291 2554 generic.go:356] "Generic (PLEG): container finished" podID="a0677508-a525-4f15-b8d4-5681080df376" containerID="c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2" exitCode=0 Jul 24 22:55:06.426407 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.426327 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" event={"ID":"a0677508-a525-4f15-b8d4-5681080df376","Type":"ContainerDied","Data":"c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2"} Jul 24 22:55:06.426407 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.426376 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" event={"ID":"a0677508-a525-4f15-b8d4-5681080df376","Type":"ContainerDied","Data":"058f4255391ea108266f397ab1aaa2a1e51b2ee65500b9c3c704134f9e7eb15a"} Jul 24 22:55:06.426407 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.426339 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f" Jul 24 22:55:06.436368 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.436349 2554 scope.go:117] "RemoveContainer" containerID="8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660" Jul 24 22:55:06.436628 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:06.436610 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660\": container with ID starting with 8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660 not found: ID does not exist" containerID="8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660" Jul 24 22:55:06.436723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.436634 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660"} err="failed to get container status \"8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660\": rpc error: code = NotFound desc = could not find container \"8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660\": container with ID starting with 8d98f5881cf9e38047b15b25351bda9852782183fe99af4047d0f7e6f5201660 not found: ID does not exist" Jul 24 22:55:06.436723 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.436649 2554 scope.go:117] "RemoveContainer" containerID="c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2" Jul 24 22:55:06.444858 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.444843 2554 scope.go:117] "RemoveContainer" containerID="c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2" Jul 24 22:55:06.445086 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:06.445070 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2\": container with ID starting with c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2 not found: ID does not exist" containerID="c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2" Jul 24 22:55:06.445135 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.445092 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2"} err="failed to get container status \"c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2\": rpc error: code = NotFound desc = could not find container \"c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2\": container with ID starting with c17cf74252fa9c24c3ef2bf2ec35f49136c1ff4f6ce59f7fb75e3fbaa878c9c2 not found: ID does not exist" Jul 24 22:55:06.450797 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.450776 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc"] Jul 24 22:55:06.454672 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.454655 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-aac4b8-679b679556-9dfqc"] Jul 24 22:55:06.465109 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.465087 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f"] Jul 24 22:55:06.467428 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:06.467404 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-aac4b8-d9679579d-q6l6f"] Jul 24 22:55:07.175869 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:07.175834 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0677508-a525-4f15-b8d4-5681080df376" path="/var/lib/kubelet/pods/a0677508-a525-4f15-b8d4-5681080df376/volumes" Jul 24 22:55:07.176258 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:07.176244 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" path="/var/lib/kubelet/pods/b4c81dbb-4e55-44c9-a8c7-55f022ef94f2/volumes" Jul 24 22:55:10.546428 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:10.546393 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll"] Jul 24 22:55:10.549076 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:10.549052 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aac4b8-smsll"] Jul 24 22:55:11.176242 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:11.176199 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55d2dece-dfc8-467d-adeb-3d4184a8461f" path="/var/lib/kubelet/pods/55d2dece-dfc8-467d-adeb-3d4184a8461f/volumes" Jul 24 22:55:17.103306 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:17.103265 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz"] Jul 24 22:55:17.103902 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:17.103589 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" containerID="cri-o://9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728" gracePeriod=30 Jul 24 22:55:22.256920 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.256883 2554 patch_prober.go:28] interesting pod/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 24 22:55:22.257259 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.256964 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 24 22:55:22.353729 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.353708 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:55:22.437933 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.437897 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-token\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438064 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.437958 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istiod-ca-cert\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438064 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.437989 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-envoy\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438064 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438004 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-certs\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438064 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438025 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-credential-socket\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438281 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438074 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkvs9\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-kube-api-access-rkvs9\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438281 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438108 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-podinfo\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438281 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438135 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-data\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438281 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438200 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-socket\") pod \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\" (UID: \"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e\") " Jul 24 22:55:22.438483 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438327 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:22.438483 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438440 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:22.438602 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438486 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:55:22.438602 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438506 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.438602 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438524 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.438930 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.438768 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:22.439472 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.439445 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:22.440253 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.440231 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:22.440393 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.440369 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:22.440489 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.440425 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-kube-api-access-rkvs9" (OuterVolumeSpecName: "kube-api-access-rkvs9") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "kube-api-access-rkvs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:22.440615 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.440595 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" (UID: "ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:55:22.509876 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.509853 2554 generic.go:356] "Generic (PLEG): container finished" podID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerID="9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728" exitCode=0 Jul 24 22:55:22.509994 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.509900 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" event={"ID":"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e","Type":"ContainerDied","Data":"9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728"} Jul 24 22:55:22.509994 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.509902 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" Jul 24 22:55:22.509994 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.509966 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz" event={"ID":"ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e","Type":"ContainerDied","Data":"9880d528bf39c3c767293c5f1e69f78d3936356063bf0f733e0959e76b62c4e3"} Jul 24 22:55:22.509994 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.509987 2554 scope.go:117] "RemoveContainer" containerID="9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728" Jul 24 22:55:22.518898 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.518879 2554 scope.go:117] "RemoveContainer" containerID="6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633" Jul 24 22:55:22.526526 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.526508 2554 scope.go:117] "RemoveContainer" containerID="9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728" Jul 24 22:55:22.526783 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:22.526764 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728\": container with ID starting with 9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728 not found: ID does not exist" containerID="9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728" Jul 24 22:55:22.526867 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.526793 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728"} err="failed to get container status \"9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728\": rpc error: code = NotFound desc = could not find container \"9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728\": container with ID starting with 9e0a85b35d137c5a6145c9f36ee465b127dd82ad404eab78f7ba23b4540c9728 not found: ID does not exist" Jul 24 22:55:22.526867 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.526817 2554 scope.go:117] "RemoveContainer" containerID="6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633" Jul 24 22:55:22.527050 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:22.527033 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633\": container with ID starting with 6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633 not found: ID does not exist" containerID="6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633" Jul 24 22:55:22.527091 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.527057 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633"} err="failed to get container status \"6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633\": rpc error: code = NotFound desc = could not find container \"6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633\": container with ID starting with 6aef24130021b853ae1ae470f185e2382c7d0a9e51cdeae0896d4fb76fd20633 not found: ID does not exist" Jul 24 22:55:22.531813 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.531792 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz"] Jul 24 22:55:22.535269 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.535251 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-aac4b8-openshift-default-559b4847d-kxbvz"] Jul 24 22:55:22.539576 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539558 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkvs9\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-kube-api-access-rkvs9\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.539656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539579 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.539656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539589 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.539656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539596 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.539656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539603 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.539656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539611 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:22.539656 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:22.539619 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:23.177477 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:23.177388 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" path="/var/lib/kubelet/pods/ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e/volumes" Jul 24 22:55:33.036008 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.035961 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl"] Jul 24 22:55:33.036495 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.036253 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" podUID="f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" containerName="payload-processing" containerID="cri-o://6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5" gracePeriod=30 Jul 24 22:55:33.041169 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.041123 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p"] Jul 24 22:55:33.041817 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.041750 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" podUID="1272087a-c75d-4bbc-8f29-37d67244063e" containerName="payload-pre-processing" containerID="cri-o://1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed" gracePeriod=30 Jul 24 22:55:33.317200 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.317176 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:55:33.319936 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.319918 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:55:33.421575 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.421545 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1272087a-c75d-4bbc-8f29-37d67244063e-plugins-config-volume\") pod \"1272087a-c75d-4bbc-8f29-37d67244063e\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " Jul 24 22:55:33.421721 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.421631 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsjts\" (UniqueName: \"kubernetes.io/projected/1272087a-c75d-4bbc-8f29-37d67244063e-kube-api-access-hsjts\") pod \"1272087a-c75d-4bbc-8f29-37d67244063e\" (UID: \"1272087a-c75d-4bbc-8f29-37d67244063e\") " Jul 24 22:55:33.421721 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.421658 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6w2d\" (UniqueName: \"kubernetes.io/projected/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-kube-api-access-w6w2d\") pod \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " Jul 24 22:55:33.421721 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.421685 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-plugins-config-volume\") pod \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\" (UID: \"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091\") " Jul 24 22:55:33.421956 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.421931 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1272087a-c75d-4bbc-8f29-37d67244063e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1272087a-c75d-4bbc-8f29-37d67244063e" (UID: "1272087a-c75d-4bbc-8f29-37d67244063e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:55:33.422092 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.422065 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" (UID: "f2b6b6e1-6177-4ad9-a9ee-629ea9d87091"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:55:33.423681 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.423659 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1272087a-c75d-4bbc-8f29-37d67244063e-kube-api-access-hsjts" (OuterVolumeSpecName: "kube-api-access-hsjts") pod "1272087a-c75d-4bbc-8f29-37d67244063e" (UID: "1272087a-c75d-4bbc-8f29-37d67244063e"). InnerVolumeSpecName "kube-api-access-hsjts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:33.423780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.423766 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-kube-api-access-w6w2d" (OuterVolumeSpecName: "kube-api-access-w6w2d") pod "f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" (UID: "f2b6b6e1-6177-4ad9-a9ee-629ea9d87091"). InnerVolumeSpecName "kube-api-access-w6w2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:33.522819 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.522797 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1272087a-c75d-4bbc-8f29-37d67244063e-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:33.522819 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.522816 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsjts\" (UniqueName: \"kubernetes.io/projected/1272087a-c75d-4bbc-8f29-37d67244063e-kube-api-access-hsjts\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:33.522940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.522826 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6w2d\" (UniqueName: \"kubernetes.io/projected/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-kube-api-access-w6w2d\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:33.522940 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.522834 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:33.569339 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.569285 2554 generic.go:356] "Generic (PLEG): container finished" podID="f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" containerID="6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5" exitCode=0 Jul 24 22:55:33.569339 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.569329 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" Jul 24 22:55:33.569475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.569384 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" event={"ID":"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091","Type":"ContainerDied","Data":"6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5"} Jul 24 22:55:33.569475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.569439 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl" event={"ID":"f2b6b6e1-6177-4ad9-a9ee-629ea9d87091","Type":"ContainerDied","Data":"c003fcff8a8dfcd0db636fc9247ab5901b0d3c5b1599b19d4973e72a8a10b4b1"} Jul 24 22:55:33.569475 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.569472 2554 scope.go:117] "RemoveContainer" containerID="6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5" Jul 24 22:55:33.570828 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.570799 2554 generic.go:356] "Generic (PLEG): container finished" podID="1272087a-c75d-4bbc-8f29-37d67244063e" containerID="1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed" exitCode=0 Jul 24 22:55:33.570917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.570832 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" Jul 24 22:55:33.570917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.570834 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" event={"ID":"1272087a-c75d-4bbc-8f29-37d67244063e","Type":"ContainerDied","Data":"1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed"} Jul 24 22:55:33.570917 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.570872 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p" event={"ID":"1272087a-c75d-4bbc-8f29-37d67244063e","Type":"ContainerDied","Data":"85ee2c56f0aa4ad98650317758b8c3e82c40f6b3f971640fbc66c33d1fb7d159"} Jul 24 22:55:33.578900 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.578866 2554 scope.go:117] "RemoveContainer" containerID="6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5" Jul 24 22:55:33.579195 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:33.579175 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5\": container with ID starting with 6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5 not found: ID does not exist" containerID="6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5" Jul 24 22:55:33.579271 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.579203 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5"} err="failed to get container status \"6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5\": rpc error: code = NotFound desc = could not find container \"6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5\": container with ID starting with 6ec7d8d654f970fbd69a05ccf8c63e959015fa4c3e849e17df312a93184c8aa5 not found: ID does not exist" Jul 24 22:55:33.579271 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.579220 2554 scope.go:117] "RemoveContainer" containerID="1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed" Jul 24 22:55:33.587358 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.587342 2554 scope.go:117] "RemoveContainer" containerID="1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed" Jul 24 22:55:33.587611 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:33.587593 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed\": container with ID starting with 1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed not found: ID does not exist" containerID="1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed" Jul 24 22:55:33.587660 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.587618 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed"} err="failed to get container status \"1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed\": rpc error: code = NotFound desc = could not find container \"1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed\": container with ID starting with 1600c0f46f391b2f88b8970becec3f8fc8c0cf75db622ed4bba1aa6c593511ed not found: ID does not exist" Jul 24 22:55:33.594130 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.594106 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p"] Jul 24 22:55:33.597192 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.597172 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-76c851-76c9b594c6-tws9p"] Jul 24 22:55:33.607073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.607053 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl"] Jul 24 22:55:33.608864 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:33.608846 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-76c851-55c86df6b6-xfvgl"] Jul 24 22:55:35.176536 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:35.176500 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1272087a-c75d-4bbc-8f29-37d67244063e" path="/var/lib/kubelet/pods/1272087a-c75d-4bbc-8f29-37d67244063e/volumes" Jul 24 22:55:35.176910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:35.176865 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" path="/var/lib/kubelet/pods/f2b6b6e1-6177-4ad9-a9ee-629ea9d87091/volumes" Jul 24 22:55:43.930558 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:43.930521 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f"] Jul 24 22:55:43.931019 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:43.930745 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" containerID="cri-o://e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d" gracePeriod=30 Jul 24 22:55:44.775128 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775092 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99"] Jul 24 22:55:44.775491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775456 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1272087a-c75d-4bbc-8f29-37d67244063e" containerName="payload-pre-processing" Jul 24 22:55:44.775491 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775478 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1272087a-c75d-4bbc-8f29-37d67244063e" containerName="payload-pre-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775501 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55d2dece-dfc8-467d-adeb-3d4184a8461f" containerName="revoke-keys" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775511 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="55d2dece-dfc8-467d-adeb-3d4184a8461f" containerName="revoke-keys" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775522 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775529 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775535 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775542 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775555 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" containerName="payload-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775560 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" containerName="payload-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775571 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" containerName="payload-pre-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775577 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" containerName="payload-pre-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775586 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0677508-a525-4f15-b8d4-5681080df376" containerName="payload-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775594 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0677508-a525-4f15-b8d4-5681080df376" containerName="payload-processing" Jul 24 22:55:44.775669 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775676 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0677508-a525-4f15-b8d4-5681080df376" containerName="payload-processing" Jul 24 22:55:44.776315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775686 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b6b6e1-6177-4ad9-a9ee-629ea9d87091" containerName="payload-processing" Jul 24 22:55:44.776315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775696 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" Jul 24 22:55:44.776315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775704 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="55d2dece-dfc8-467d-adeb-3d4184a8461f" containerName="revoke-keys" Jul 24 22:55:44.776315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775714 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab68bc2a-c3d4-421c-9b8e-2a17a3186e7e" containerName="istio-proxy" Jul 24 22:55:44.776315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775722 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4c81dbb-4e55-44c9-a8c7-55f022ef94f2" containerName="payload-pre-processing" Jul 24 22:55:44.776315 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.775732 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1272087a-c75d-4bbc-8f29-37d67244063e" containerName="payload-pre-processing" Jul 24 22:55:44.778864 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.778846 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.781630 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.781609 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-499dd1-openshift-default-dockercfg-s6dtm\"" Jul 24 22:55:44.794730 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.794707 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99"] Jul 24 22:55:44.811910 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.811883 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-certs\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.811919 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb80707-e293-4a11-ab07-a5700d3abc57-istiod-ca-cert\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812011 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.811952 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-socket\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812107 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.812002 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-token\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812107 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.812081 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-envoy\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812206 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.812143 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-credential-socket\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812270 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.812248 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjfg8\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-kube-api-access-vjfg8\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812310 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.812295 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-podinfo\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.812369 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.812350 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-data\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.913627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913592 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-data\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.913627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913632 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-certs\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.913870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913652 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb80707-e293-4a11-ab07-a5700d3abc57-istiod-ca-cert\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.913870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913674 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-socket\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.913870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913777 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-token\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.913870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913817 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-envoy\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913909 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-credential-socket\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.913986 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjfg8\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-kube-api-access-vjfg8\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.914025 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-podinfo\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914075 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.914049 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-certs\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914321 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.914084 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-data\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914321 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.914201 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-socket\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914436 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.914334 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-credential-socket\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.914535 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.914515 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb80707-e293-4a11-ab07-a5700d3abc57-istiod-ca-cert\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.916116 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.916095 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-envoy\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.916624 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.916605 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-podinfo\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.926338 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.926320 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjfg8\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-kube-api-access-vjfg8\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:44.926445 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:44.926390 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-token\") pod \"e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:45.088978 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.088888 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:45.248879 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.248850 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99"] Jul 24 22:55:45.250054 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:55:45.250030 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb80707_e293_4a11_ab07_a5700d3abc57.slice/crio-1921bb7e67abd1b56602a6d9e2fea733f84393c06b034e8c0ef3b4d63b2ded0f WatchSource:0}: Error finding container 1921bb7e67abd1b56602a6d9e2fea733f84393c06b034e8c0ef3b4d63b2ded0f: Status 404 returned error can't find the container with id 1921bb7e67abd1b56602a6d9e2fea733f84393c06b034e8c0ef3b4d63b2ded0f Jul 24 22:55:45.252048 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.251994 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:55:45.252113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.252074 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:55:45.252113 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.252106 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:55:45.632975 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.632938 2554 patch_prober.go:28] interesting pod/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 22:55:45.633123 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.632999 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 22:55:45.638809 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.638782 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" event={"ID":"fcb80707-e293-4a11-ab07-a5700d3abc57","Type":"ContainerStarted","Data":"64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708"} Jul 24 22:55:45.638951 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.638857 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" event={"ID":"fcb80707-e293-4a11-ab07-a5700d3abc57","Type":"ContainerStarted","Data":"1921bb7e67abd1b56602a6d9e2fea733f84393c06b034e8c0ef3b4d63b2ded0f"} Jul 24 22:55:45.664316 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:45.664272 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podStartSLOduration=1.664258642 podStartE2EDuration="1.664258642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:45.663863649 +0000 UTC m=+2443.050681731" watchObservedRunningTime="2026-07-24 22:55:45.664258642 +0000 UTC m=+2443.051076734" Jul 24 22:55:46.089398 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:46.089363 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:46.094544 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:46.094519 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:46.644144 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:46.644117 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:46.645307 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:46.645278 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:55:48.305387 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.305357 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k"] Jul 24 22:55:48.308422 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.308401 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.310884 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.310866 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-499dd1-dockercfg-496tx\"" Jul 24 22:55:48.310961 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.310866 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-499dd1\"" Jul 24 22:55:48.318741 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.318718 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k"] Jul 24 22:55:48.341468 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.341447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.341585 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.341483 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsw2n\" (UniqueName: \"kubernetes.io/projected/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-kube-api-access-vsw2n\") pod \"payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.351731 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.351706 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx"] Jul 24 22:55:48.354916 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.354899 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.372926 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.372905 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx"] Jul 24 22:55:48.442488 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.442460 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.442659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.442518 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kp77\" (UniqueName: \"kubernetes.io/projected/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-kube-api-access-8kp77\") pod \"payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.442659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.442549 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.442659 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.442577 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsw2n\" (UniqueName: \"kubernetes.io/projected/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-kube-api-access-vsw2n\") pod \"payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.443301 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.443283 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.451442 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.451418 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsw2n\" (UniqueName: \"kubernetes.io/projected/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-kube-api-access-vsw2n\") pod \"payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.543375 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.543345 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.543502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.543395 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kp77\" (UniqueName: \"kubernetes.io/projected/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-kube-api-access-8kp77\") pod \"payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.543965 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.543946 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.555770 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.555705 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kp77\" (UniqueName: \"kubernetes.io/projected/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-kube-api-access-8kp77\") pod \"payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.620320 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.620299 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:48.663590 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.663560 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:48.771805 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.771728 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k"] Jul 24 22:55:48.773401 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:55:48.773374 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77fc5a60_3070_4904_a9c1_c8a4b6d9d123.slice/crio-9e541cecb5673dde72927b802fddc039736e1dbf5deddd75de05f62a5efbcbb4 WatchSource:0}: Error finding container 9e541cecb5673dde72927b802fddc039736e1dbf5deddd75de05f62a5efbcbb4: Status 404 returned error can't find the container with id 9e541cecb5673dde72927b802fddc039736e1dbf5deddd75de05f62a5efbcbb4 Jul 24 22:55:48.816609 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:55:48.816577 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0bee41_0a14_4b10_a36d_8c9ef708dd22.slice/crio-91ca37ebc1dc1d639fa8c749110a1270493b53c9025ba70255479ef27e0bd6b4 WatchSource:0}: Error finding container 91ca37ebc1dc1d639fa8c749110a1270493b53c9025ba70255479ef27e0bd6b4: Status 404 returned error can't find the container with id 91ca37ebc1dc1d639fa8c749110a1270493b53c9025ba70255479ef27e0bd6b4 Jul 24 22:55:48.817350 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:48.817324 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx"] Jul 24 22:55:49.169163 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.169134 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:55:49.248136 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248104 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-credential-socket\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248136 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248136 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-socket\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248166 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-data\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248189 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2182204-442b-4a2f-b062-d1349d328748-istiod-ca-cert\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248215 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7575\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-kube-api-access-t7575\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248235 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-envoy\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248277 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-certs\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248306 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2182204-442b-4a2f-b062-d1349d328748-istio-podinfo\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248329 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-istio-token\") pod \"a2182204-442b-4a2f-b062-d1349d328748\" (UID: \"a2182204-442b-4a2f-b062-d1349d328748\") " Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248390 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248468 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248556 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248657 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248675 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248688 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.248846 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.248740 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2182204-442b-4a2f-b062-d1349d328748-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 22:55:49.249444 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.249369 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:49.250948 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.250914 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:49.251073 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.251003 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2182204-442b-4a2f-b062-d1349d328748-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 22:55:49.251329 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.251299 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-kube-api-access-t7575" (OuterVolumeSpecName: "kube-api-access-t7575") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "kube-api-access-t7575". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 22:55:49.251766 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.251743 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2182204-442b-4a2f-b062-d1349d328748" (UID: "a2182204-442b-4a2f-b062-d1349d328748"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 22:55:49.349233 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.349211 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.349233 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.349232 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2182204-442b-4a2f-b062-d1349d328748-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.349542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.349242 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7575\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-kube-api-access-t7575\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.349542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.349251 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2182204-442b-4a2f-b062-d1349d328748-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.349542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.349260 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2182204-442b-4a2f-b062-d1349d328748-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.349542 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.349268 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2182204-442b-4a2f-b062-d1349d328748-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 22:55:49.662423 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.662391 2554 generic.go:356] "Generic (PLEG): container finished" podID="a2182204-442b-4a2f-b062-d1349d328748" containerID="e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d" exitCode=0 Jul 24 22:55:49.662569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.662464 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" Jul 24 22:55:49.662569 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.662497 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" event={"ID":"a2182204-442b-4a2f-b062-d1349d328748","Type":"ContainerDied","Data":"e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d"} Jul 24 22:55:49.662702 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.662587 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f" event={"ID":"a2182204-442b-4a2f-b062-d1349d328748","Type":"ContainerDied","Data":"6e23c1ff983acd5eddf1089900d2ef3eb9fe3f1c3ec880026b35b4d339a785a5"} Jul 24 22:55:49.662702 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.662620 2554 scope.go:117] "RemoveContainer" containerID="e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d" Jul 24 22:55:49.664468 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.664194 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" event={"ID":"ed0bee41-0a14-4b10-a36d-8c9ef708dd22","Type":"ContainerStarted","Data":"a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092"} Jul 24 22:55:49.664468 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.664241 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" event={"ID":"ed0bee41-0a14-4b10-a36d-8c9ef708dd22","Type":"ContainerStarted","Data":"91ca37ebc1dc1d639fa8c749110a1270493b53c9025ba70255479ef27e0bd6b4"} Jul 24 22:55:49.664468 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.664293 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:55:49.666122 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.666100 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" event={"ID":"77fc5a60-3070-4904-a9c1-c8a4b6d9d123","Type":"ContainerStarted","Data":"b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0"} Jul 24 22:55:49.666260 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.666143 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" event={"ID":"77fc5a60-3070-4904-a9c1-c8a4b6d9d123","Type":"ContainerStarted","Data":"9e541cecb5673dde72927b802fddc039736e1dbf5deddd75de05f62a5efbcbb4"} Jul 24 22:55:49.666260 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.666212 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:55:49.673028 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.673011 2554 scope.go:117] "RemoveContainer" containerID="fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e" Jul 24 22:55:49.680753 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.680703 2554 scope.go:117] "RemoveContainer" containerID="e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d" Jul 24 22:55:49.681118 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:49.681094 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d\": container with ID starting with e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d not found: ID does not exist" containerID="e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d" Jul 24 22:55:49.681319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.681129 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d"} err="failed to get container status \"e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d\": rpc error: code = NotFound desc = could not find container \"e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d\": container with ID starting with e4531b7b5a47b98f7a3b943dfec4a1ac267b7febd7644793c807436d66fcba1d not found: ID does not exist" Jul 24 22:55:49.681319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.681179 2554 scope.go:117] "RemoveContainer" containerID="fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e" Jul 24 22:55:49.681562 ip-10-0-143-53 kubenswrapper[2554]: E0724 22:55:49.681536 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e\": container with ID starting with fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e not found: ID does not exist" containerID="fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e" Jul 24 22:55:49.681627 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.681569 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e"} err="failed to get container status \"fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e\": rpc error: code = NotFound desc = could not find container \"fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e\": container with ID starting with fcc727076b541e6528e53c4db95c147cabd50feb49a9ccc3130504c832b03f4e not found: ID does not exist" Jul 24 22:55:49.681873 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.681835 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" podStartSLOduration=1.681823521 podStartE2EDuration="1.681823521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:49.680972469 +0000 UTC m=+2447.067790517" watchObservedRunningTime="2026-07-24 22:55:49.681823521 +0000 UTC m=+2447.068641566" Jul 24 22:55:49.695870 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.695787 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" podStartSLOduration=1.6957741670000002 podStartE2EDuration="1.695774167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:49.695613086 +0000 UTC m=+2447.082431157" watchObservedRunningTime="2026-07-24 22:55:49.695774167 +0000 UTC m=+2447.082592217" Jul 24 22:55:49.711706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.711652 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f"] Jul 24 22:55:49.713419 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:49.713401 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-76c851-openshift-default-6d484b9766-jgr8f"] Jul 24 22:55:51.175616 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:51.175585 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2182204-442b-4a2f-b062-d1349d328748" path="/var/lib/kubelet/pods/a2182204-442b-4a2f-b062-d1349d328748/volumes" Jul 24 22:55:52.651787 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.651757 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d"] Jul 24 22:55:52.652279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.652085 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" Jul 24 22:55:52.652279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.652100 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" Jul 24 22:55:52.652279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.652137 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" Jul 24 22:55:52.652279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.652143 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" Jul 24 22:55:52.652279 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.652231 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" Jul 24 22:55:52.656532 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.656512 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.659015 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.658996 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e15986-openshift-default-dockercfg-tzg6g\"" Jul 24 22:55:52.667845 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.667821 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d"] Jul 24 22:55:52.774502 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774461 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-credential-socket\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774708 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774530 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-data\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774708 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774578 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0035146-9042-4f88-9388-d728b78bd60f-istiod-ca-cert\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774708 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774627 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-socket\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774708 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774665 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxpdt\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-kube-api-access-lxpdt\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774776 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-envoy\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774804 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-istio-token\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774836 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-certs\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.774934 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.774910 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0035146-9042-4f88-9388-d728b78bd60f-istio-podinfo\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876277 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876233 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0035146-9042-4f88-9388-d728b78bd60f-istiod-ca-cert\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876410 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876285 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-socket\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876410 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876399 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxpdt\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-kube-api-access-lxpdt\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876527 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876459 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-envoy\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876527 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876485 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-istio-token\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876637 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876525 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-certs\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876637 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876588 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0035146-9042-4f88-9388-d728b78bd60f-istio-podinfo\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876637 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-credential-socket\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876643 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-data\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876780 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876675 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-socket\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.876963 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876941 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-credential-socket\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.877022 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.876992 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-data\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.877076 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.877041 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0035146-9042-4f88-9388-d728b78bd60f-istiod-ca-cert\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.877173 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.877127 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-certs\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.878905 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.878882 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-envoy\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.879010 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.878992 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0035146-9042-4f88-9388-d728b78bd60f-istio-podinfo\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.884571 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.884548 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxpdt\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-kube-api-access-lxpdt\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.884678 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.884659 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-istio-token\") pod \"e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:52.969145 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:52.969091 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:53.101302 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.101277 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d"] Jul 24 22:55:53.103208 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:55:53.103138 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0035146_9042_4f88_9388_d728b78bd60f.slice/crio-35847a4ab58b61ec5433d85e186003c009e689089516d0b77d8b7abebd6b3e4e WatchSource:0}: Error finding container 35847a4ab58b61ec5433d85e186003c009e689089516d0b77d8b7abebd6b3e4e: Status 404 returned error can't find the container with id 35847a4ab58b61ec5433d85e186003c009e689089516d0b77d8b7abebd6b3e4e Jul 24 22:55:53.105488 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.105455 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:55:53.105573 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.105518 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:55:53.105573 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.105546 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:55:53.690587 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.690554 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" event={"ID":"b0035146-9042-4f88-9388-d728b78bd60f","Type":"ContainerStarted","Data":"debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46"} Jul 24 22:55:53.691026 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.690603 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" event={"ID":"b0035146-9042-4f88-9388-d728b78bd60f","Type":"ContainerStarted","Data":"35847a4ab58b61ec5433d85e186003c009e689089516d0b77d8b7abebd6b3e4e"} Jul 24 22:55:53.711061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.711012 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" podStartSLOduration=1.71099731 podStartE2EDuration="1.71099731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:53.708863578 +0000 UTC m=+2451.095681648" watchObservedRunningTime="2026-07-24 22:55:53.71099731 +0000 UTC m=+2451.097815355" Jul 24 22:55:53.969353 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:53.969280 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:54.980413 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:54.980379 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:55.704789 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:55.704757 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:55.705767 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:55.705747 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:55:56.346187 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.346136 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6"] Jul 24 22:55:56.346696 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.346676 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2182204-442b-4a2f-b062-d1349d328748" containerName="istio-proxy" Jul 24 22:55:56.349791 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.349770 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.358175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.354713 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-e15986-dockercfg-kcbwx\"" Jul 24 22:55:56.358175 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.354946 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-e15986\"" Jul 24 22:55:56.368712 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.366102 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6"] Jul 24 22:55:56.398026 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.398000 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg"] Jul 24 22:55:56.401286 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.401266 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.406319 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.406291 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2912b3cb-3ec7-4159-b63c-edc7391fa06a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.406423 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.406393 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkvrc\" (UniqueName: \"kubernetes.io/projected/2912b3cb-3ec7-4159-b63c-edc7391fa06a-kube-api-access-tkvrc\") pod \"payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.414244 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.414221 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg"] Jul 24 22:55:56.507257 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.507226 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.507470 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.507307 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2912b3cb-3ec7-4159-b63c-edc7391fa06a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.507470 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.507367 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkvrc\" (UniqueName: \"kubernetes.io/projected/2912b3cb-3ec7-4159-b63c-edc7391fa06a-kube-api-access-tkvrc\") pod \"payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.507470 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.507386 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f62r\" (UniqueName: \"kubernetes.io/projected/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-kube-api-access-4f62r\") pod \"payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.514035 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.508525 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2912b3cb-3ec7-4159-b63c-edc7391fa06a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.517097 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.517067 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkvrc\" (UniqueName: \"kubernetes.io/projected/2912b3cb-3ec7-4159-b63c-edc7391fa06a-kube-api-access-tkvrc\") pod \"payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.607999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.607917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f62r\" (UniqueName: \"kubernetes.io/projected/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-kube-api-access-4f62r\") pod \"payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.607999 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.607980 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.608781 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.608761 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.620404 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.620378 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f62r\" (UniqueName: \"kubernetes.io/projected/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-kube-api-access-4f62r\") pod \"payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.680071 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.680045 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:56.710860 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.710837 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:55:56.868756 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:55:56.868701 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2912b3cb_3ec7_4159_b63c_edc7391fa06a.slice/crio-c155bc918d20468cfccb6414ee965f8cae45096d58c7328f4940eda4274ec1f8 WatchSource:0}: Error finding container c155bc918d20468cfccb6414ee965f8cae45096d58c7328f4940eda4274ec1f8: Status 404 returned error can't find the container with id c155bc918d20468cfccb6414ee965f8cae45096d58c7328f4940eda4274ec1f8 Jul 24 22:55:56.879500 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.879383 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6"] Jul 24 22:55:56.923050 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:56.922964 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg"] Jul 24 22:55:56.925546 ip-10-0-143-53 kubenswrapper[2554]: W0724 22:55:56.925525 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93c6f9f_cec2_40b9_b722_3e9e4fb47dc1.slice/crio-bdaa039583463cabd68f74683fa025c94396f8f6bf428f95c9337e9c83aa8174 WatchSource:0}: Error finding container bdaa039583463cabd68f74683fa025c94396f8f6bf428f95c9337e9c83aa8174: Status 404 returned error can't find the container with id bdaa039583463cabd68f74683fa025c94396f8f6bf428f95c9337e9c83aa8174 Jul 24 22:55:57.716577 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.716541 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" event={"ID":"2912b3cb-3ec7-4159-b63c-edc7391fa06a","Type":"ContainerStarted","Data":"0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c"} Jul 24 22:55:57.717014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.716606 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" event={"ID":"2912b3cb-3ec7-4159-b63c-edc7391fa06a","Type":"ContainerStarted","Data":"c155bc918d20468cfccb6414ee965f8cae45096d58c7328f4940eda4274ec1f8"} Jul 24 22:55:57.717014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.716706 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:55:57.718174 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.718138 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" event={"ID":"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1","Type":"ContainerStarted","Data":"80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677"} Jul 24 22:55:57.718316 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.718187 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" event={"ID":"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1","Type":"ContainerStarted","Data":"bdaa039583463cabd68f74683fa025c94396f8f6bf428f95c9337e9c83aa8174"} Jul 24 22:55:57.733642 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.733601 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" podStartSLOduration=1.7335873240000002 podStartE2EDuration="1.733587324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:57.73125874 +0000 UTC m=+2455.118076788" watchObservedRunningTime="2026-07-24 22:55:57.733587324 +0000 UTC m=+2455.120405369" Jul 24 22:55:57.746334 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:57.746293 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" podStartSLOduration=1.7462801799999998 podStartE2EDuration="1.74628018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 22:55:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 22:55:57.745299603 +0000 UTC m=+2455.132117674" watchObservedRunningTime="2026-07-24 22:55:57.74628018 +0000 UTC m=+2455.133098225" Jul 24 22:55:58.725268 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:55:58.725231 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:56:00.677746 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:00.677689 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 22:56:00.678853 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:00.678794 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 22:56:01.645771 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.645743 2554 patch_prober.go:28] interesting pod/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 22:56:01.645971 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.645797 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 22:56:01.743773 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.743739 2554 generic.go:356] "Generic (PLEG): container finished" podID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerID="64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708" exitCode=0 Jul 24 22:56:01.744262 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.743799 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" event={"ID":"fcb80707-e293-4a11-ab07-a5700d3abc57","Type":"ContainerDied","Data":"64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708"} Jul 24 22:56:01.744327 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.744312 2554 scope.go:117] "RemoveContainer" containerID="64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708" Jul 24 22:56:01.745376 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.745340 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:56:01.745479 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.745426 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:56:01.745479 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:01.745471 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:56:02.752437 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:02.751531 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" event={"ID":"fcb80707-e293-4a11-ab07-a5700d3abc57","Type":"ContainerStarted","Data":"d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468"} Jul 24 22:56:03.089815 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:03.089716 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:56:03.090143 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:03.090121 2554 patch_prober.go:28] interesting pod/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 22:56:03.090360 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:03.090334 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 22:56:04.089671 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:04.089633 2554 patch_prober.go:28] interesting pod/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 22:56:04.090061 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:04.089685 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 22:56:05.090339 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:05.090301 2554 patch_prober.go:28] interesting pod/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 22:56:05.090844 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:05.090362 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 22:56:06.089678 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:06.089641 2554 patch_prober.go:28] interesting pod/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 24 22:56:06.089855 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:06.089733 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 24 22:56:07.093243 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.093215 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:56:07.093648 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.093550 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:56:07.094539 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.094521 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 22:56:07.790987 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.790951 2554 generic.go:356] "Generic (PLEG): container finished" podID="b0035146-9042-4f88-9388-d728b78bd60f" containerID="debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46" exitCode=0 Jul 24 22:56:07.791221 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.791025 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" event={"ID":"b0035146-9042-4f88-9388-d728b78bd60f","Type":"ContainerDied","Data":"debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46"} Jul 24 22:56:07.791550 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.791533 2554 scope.go:117] "RemoveContainer" containerID="debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46" Jul 24 22:56:07.792505 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.792480 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:56:07.792565 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.792543 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:56:07.792611 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:07.792576 2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 22:56:08.727014 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:08.726986 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 22:56:08.797896 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:08.797864 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" event={"ID":"b0035146-9042-4f88-9388-d728b78bd60f","Type":"ContainerStarted","Data":"8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f"} Jul 24 22:56:08.970304 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:08.970273 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:56:08.971706 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:08.971676 2554 patch_prober.go:28] interesting pod/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 24 22:56:08.971929 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:08.971899 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 24 22:56:09.733334 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:09.733304 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 22:56:09.969867 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:09.969828 2554 patch_prober.go:28] interesting pod/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 24 22:56:09.970200 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:09.970136 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 24 22:56:10.705577 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:10.705546 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:56:10.969791 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:10.969704 2554 patch_prober.go:28] interesting pod/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 24 22:56:10.970367 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:10.970325 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 24 22:56:11.969942 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:11.969899 2554 patch_prober.go:28] interesting pod/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 24 22:56:11.970488 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:11.969966 2554 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 24 22:56:12.973359 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:12.973331 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 22:56:12.974526 ip-10-0-143-53 kubenswrapper[2554]: I0724 22:56:12.974504 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 23:00:00.134370 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.134272 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4"] Jul 24 23:00:00.137675 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.137654 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc"] Jul 24 23:00:00.137796 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.137773 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" Jul 24 23:00:00.140279 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.140251 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vcbdx\"" Jul 24 23:00:00.140720 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.140704 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" Jul 24 23:00:00.145324 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.145299 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4"] Jul 24 23:00:00.148381 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.148337 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc"] Jul 24 23:00:00.152167 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.152131 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" Jul 24 23:00:00.159521 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.159497 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" Jul 24 23:00:00.298387 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.298353 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4"] Jul 24 23:00:00.300312 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:00:00.300257 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd577c762_5323_4a98_87c0_df7fcee8f7b0.slice/crio-0775a2b166d66a7541a1fc701b7c15f3ec8f603c6f354b0453d54ab8b67307a6 WatchSource:0}: Error finding container 0775a2b166d66a7541a1fc701b7c15f3ec8f603c6f354b0453d54ab8b67307a6: Status 404 returned error can't find the container with id 0775a2b166d66a7541a1fc701b7c15f3ec8f603c6f354b0453d54ab8b67307a6 Jul 24 23:00:00.331905 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.331866 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc"] Jul 24 23:00:00.336772 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:00:00.336749 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57710237_1f7b_45ac_aac7_a599d4eb1ecb.slice/crio-b2fedc27ecb244fa82f30c8982ffd32b2e0054a6f013c7b7b272c4954e8dfca4 WatchSource:0}: Error finding container b2fedc27ecb244fa82f30c8982ffd32b2e0054a6f013c7b7b272c4954e8dfca4: Status 404 returned error can't find the container with id b2fedc27ecb244fa82f30c8982ffd32b2e0054a6f013c7b7b272c4954e8dfca4 Jul 24 23:00:00.985599 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.985562 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" event={"ID":"d577c762-5323-4a98-87c0-df7fcee8f7b0","Type":"ContainerStarted","Data":"d7c35c4e98005c73c0f11f841c14a2163fc9f5487ae2fd56328d89110006e348"} Jul 24 23:00:00.985770 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.985626 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" event={"ID":"d577c762-5323-4a98-87c0-df7fcee8f7b0","Type":"ContainerStarted","Data":"0775a2b166d66a7541a1fc701b7c15f3ec8f603c6f354b0453d54ab8b67307a6"} Jul 24 23:00:00.987024 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.987001 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" event={"ID":"57710237-1f7b-45ac-aac7-a599d4eb1ecb","Type":"ContainerStarted","Data":"bf9ab3e25afee1523b8874423230b2fd7cf8e67c896bd2eea3b5113e76d23d56"} Jul 24 23:00:00.987171 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:00.987040 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" event={"ID":"57710237-1f7b-45ac-aac7-a599d4eb1ecb","Type":"ContainerStarted","Data":"b2fedc27ecb244fa82f30c8982ffd32b2e0054a6f013c7b7b272c4954e8dfca4"} Jul 24 23:00:01.002165 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:01.002101 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" podStartSLOduration=1.002086979 podStartE2EDuration="1.002086979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:01.000315775 +0000 UTC m=+2698.387133822" watchObservedRunningTime="2026-07-24 23:00:01.002086979 +0000 UTC m=+2698.388905025" Jul 24 23:00:01.019082 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:01.019036 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" podStartSLOduration=1.019024888 podStartE2EDuration="1.019024888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:00:01.017026926 +0000 UTC m=+2698.403844972" watchObservedRunningTime="2026-07-24 23:00:01.019024888 +0000 UTC m=+2698.405842934" Jul 24 23:00:01.994573 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:01.994538 2554 generic.go:356] "Generic (PLEG): container finished" podID="57710237-1f7b-45ac-aac7-a599d4eb1ecb" containerID="bf9ab3e25afee1523b8874423230b2fd7cf8e67c896bd2eea3b5113e76d23d56" exitCode=0 Jul 24 23:00:01.995067 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:01.994629 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" event={"ID":"57710237-1f7b-45ac-aac7-a599d4eb1ecb","Type":"ContainerDied","Data":"bf9ab3e25afee1523b8874423230b2fd7cf8e67c896bd2eea3b5113e76d23d56"} Jul 24 23:00:01.996145 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:01.996123 2554 generic.go:356] "Generic (PLEG): container finished" podID="d577c762-5323-4a98-87c0-df7fcee8f7b0" containerID="d7c35c4e98005c73c0f11f841c14a2163fc9f5487ae2fd56328d89110006e348" exitCode=0 Jul 24 23:00:01.996282 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:01.996191 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" event={"ID":"d577c762-5323-4a98-87c0-df7fcee8f7b0","Type":"ContainerDied","Data":"d7c35c4e98005c73c0f11f841c14a2163fc9f5487ae2fd56328d89110006e348"} Jul 24 23:00:03.133606 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:03.133582 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" Jul 24 23:00:03.160054 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:03.160025 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" Jul 24 23:00:03.453776 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:03.453747 2554 scope.go:117] "RemoveContainer" containerID="729e9242ad070f11047658759a3bb3ea7bd134262b63a4e7606ae233063f0f70" Jul 24 23:00:04.006954 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:04.006928 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" Jul 24 23:00:04.007127 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:04.006967 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc" event={"ID":"57710237-1f7b-45ac-aac7-a599d4eb1ecb","Type":"ContainerDied","Data":"b2fedc27ecb244fa82f30c8982ffd32b2e0054a6f013c7b7b272c4954e8dfca4"} Jul 24 23:00:04.007127 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:04.007045 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2fedc27ecb244fa82f30c8982ffd32b2e0054a6f013c7b7b272c4954e8dfca4" Jul 24 23:00:04.008284 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:04.008256 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" Jul 24 23:00:04.008378 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:04.008255 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4" event={"ID":"d577c762-5323-4a98-87c0-df7fcee8f7b0","Type":"ContainerDied","Data":"0775a2b166d66a7541a1fc701b7c15f3ec8f603c6f354b0453d54ab8b67307a6"} Jul 24 23:00:04.008442 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:00:04.008378 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0775a2b166d66a7541a1fc701b7c15f3ec8f603c6f354b0453d54ab8b67307a6" Jul 24 23:01:03.475943 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:01:03.475911 2554 scope.go:117] "RemoveContainer" containerID="d99a7b77ea0afd1839ae0acf60d1a3b5df814664257e3f4c6441cfcab97dd9ba" Jul 24 23:02:12.686115 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686080 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88"] Jul 24 23:02:12.686548 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686426 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57710237-1f7b-45ac-aac7-a599d4eb1ecb" containerName="cleanup" Jul 24 23:02:12.686548 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686437 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="57710237-1f7b-45ac-aac7-a599d4eb1ecb" containerName="cleanup" Jul 24 23:02:12.686548 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686446 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d577c762-5323-4a98-87c0-df7fcee8f7b0" containerName="cleanup" Jul 24 23:02:12.686548 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686451 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d577c762-5323-4a98-87c0-df7fcee8f7b0" containerName="cleanup" Jul 24 23:02:12.686548 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686507 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="57710237-1f7b-45ac-aac7-a599d4eb1ecb" containerName="cleanup" Jul 24 23:02:12.686548 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.686517 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d577c762-5323-4a98-87c0-df7fcee8f7b0" containerName="cleanup" Jul 24 23:02:12.689579 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.689564 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.692224 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.692200 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e3f0a9\"" Jul 24 23:02:12.692480 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.692466 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e3f0a9-dockercfg-t5nh9\"" Jul 24 23:02:12.702483 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.702462 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88"] Jul 24 23:02:12.711874 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.711847 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm"] Jul 24 23:02:12.715184 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.715166 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:12.723749 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.723728 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm"] Jul 24 23:02:12.760443 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.760413 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cfdc8ce-4fca-4318-af68-cba2f86e3932-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.760629 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.760452 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjfmz\" (UniqueName: \"kubernetes.io/projected/1cfdc8ce-4fca-4318-af68-cba2f86e3932-kube-api-access-kjfmz\") pod \"payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.861058 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.861016 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723b83ef-3f72-4ae0-a8dc-6027cc515325-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:12.861281 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.861094 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cfdc8ce-4fca-4318-af68-cba2f86e3932-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.861281 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.861115 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjfmz\" (UniqueName: \"kubernetes.io/projected/1cfdc8ce-4fca-4318-af68-cba2f86e3932-kube-api-access-kjfmz\") pod \"payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.861281 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.861167 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzqxf\" (UniqueName: \"kubernetes.io/projected/723b83ef-3f72-4ae0-a8dc-6027cc515325-kube-api-access-xzqxf\") pod \"payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:12.862020 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.861996 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cfdc8ce-4fca-4318-af68-cba2f86e3932-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.869890 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.869863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjfmz\" (UniqueName: \"kubernetes.io/projected/1cfdc8ce-4fca-4318-af68-cba2f86e3932-kube-api-access-kjfmz\") pod \"payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:12.962241 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.962115 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzqxf\" (UniqueName: \"kubernetes.io/projected/723b83ef-3f72-4ae0-a8dc-6027cc515325-kube-api-access-xzqxf\") pod \"payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:12.962241 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.962199 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723b83ef-3f72-4ae0-a8dc-6027cc515325-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:12.962877 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.962856 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723b83ef-3f72-4ae0-a8dc-6027cc515325-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:12.970715 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:12.970687 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzqxf\" (UniqueName: \"kubernetes.io/projected/723b83ef-3f72-4ae0-a8dc-6027cc515325-kube-api-access-xzqxf\") pod \"payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:13.002078 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.002044 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:13.026634 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.026607 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:13.384206 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.384181 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88"] Jul 24 23:02:13.385306 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:02:13.385279 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cfdc8ce_4fca_4318_af68_cba2f86e3932.slice/crio-fd828641e9dd06b55a2e2a52bc77e882b6effae976e7cc2cfd43efc467e5351c WatchSource:0}: Error finding container fd828641e9dd06b55a2e2a52bc77e882b6effae976e7cc2cfd43efc467e5351c: Status 404 returned error can't find the container with id fd828641e9dd06b55a2e2a52bc77e882b6effae976e7cc2cfd43efc467e5351c Jul 24 23:02:13.412657 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.412633 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm"] Jul 24 23:02:13.414621 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:02:13.414598 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723b83ef_3f72_4ae0_a8dc_6027cc515325.slice/crio-9816e4ff27f6eb57db7dabfa439635c8efbe70f3fcc95c318795d8d0605f66a6 WatchSource:0}: Error finding container 9816e4ff27f6eb57db7dabfa439635c8efbe70f3fcc95c318795d8d0605f66a6: Status 404 returned error can't find the container with id 9816e4ff27f6eb57db7dabfa439635c8efbe70f3fcc95c318795d8d0605f66a6 Jul 24 23:02:13.709098 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.709068 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" event={"ID":"723b83ef-3f72-4ae0-a8dc-6027cc515325","Type":"ContainerStarted","Data":"89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5"} Jul 24 23:02:13.709580 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.709119 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" event={"ID":"723b83ef-3f72-4ae0-a8dc-6027cc515325","Type":"ContainerStarted","Data":"9816e4ff27f6eb57db7dabfa439635c8efbe70f3fcc95c318795d8d0605f66a6"} Jul 24 23:02:13.709580 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.709144 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:13.710661 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.710636 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" event={"ID":"1cfdc8ce-4fca-4318-af68-cba2f86e3932","Type":"ContainerStarted","Data":"61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5"} Jul 24 23:02:13.710794 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.710678 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" event={"ID":"1cfdc8ce-4fca-4318-af68-cba2f86e3932","Type":"ContainerStarted","Data":"fd828641e9dd06b55a2e2a52bc77e882b6effae976e7cc2cfd43efc467e5351c"} Jul 24 23:02:13.710794 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.710706 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:13.731558 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.731512 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" podStartSLOduration=1.731497734 podStartE2EDuration="1.731497734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:13.729503976 +0000 UTC m=+2831.116322021" watchObservedRunningTime="2026-07-24 23:02:13.731497734 +0000 UTC m=+2831.118315774" Jul 24 23:02:13.748577 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:13.748528 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" podStartSLOduration=1.748514618 podStartE2EDuration="1.748514618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:13.74612024 +0000 UTC m=+2831.132938303" watchObservedRunningTime="2026-07-24 23:02:13.748514618 +0000 UTC m=+2831.135332664" Jul 24 23:02:24.717574 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:24.717540 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:02:24.717949 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:24.717618 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:02:31.992663 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:31.992618 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv"] Jul 24 23:02:31.997770 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:31.997753 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx"] Jul 24 23:02:31.997899 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:31.997881 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.001109 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.001084 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:32.001860 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:02:32.001819 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins-e2e-ipp-b-c6e95b\" is forbidden: User \"system:node:ip-10-0-143-53.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-53.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c6e95b\"" type="*v1.ConfigMap" Jul 24 23:02:32.001860 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:02:32.001817 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" is forbidden: User \"system:node:ip-10-0-143-53.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-53.ec2.internal' and this object" podUID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.009729 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.009709 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c6e95b-dockercfg-rr4vs\"" Jul 24 23:02:32.017012 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.016992 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx"] Jul 24 23:02:32.021064 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.021039 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:32.021192 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.021073 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.021192 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.021091 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ggwr\" (UniqueName: \"kubernetes.io/projected/7fe91845-9a01-43ed-a1c9-bab12f0498d7-kube-api-access-5ggwr\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:32.021316 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.021256 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92s5p\" (UniqueName: \"kubernetes.io/projected/ec4cda66-cfcc-42d7-91ae-ea9958214df5-kube-api-access-92s5p\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.022778 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.022759 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv"] Jul 24 23:02:32.122218 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.122186 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92s5p\" (UniqueName: \"kubernetes.io/projected/ec4cda66-cfcc-42d7-91ae-ea9958214df5-kube-api-access-92s5p\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.122414 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.122259 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:32.122414 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.122296 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.122414 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.122321 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ggwr\" (UniqueName: \"kubernetes.io/projected/7fe91845-9a01-43ed-a1c9-bab12f0498d7-kube-api-access-5ggwr\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:32.153331 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.153305 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92s5p\" (UniqueName: \"kubernetes.io/projected/ec4cda66-cfcc-42d7-91ae-ea9958214df5-kube-api-access-92s5p\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:32.153478 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:32.153434 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ggwr\" (UniqueName: \"kubernetes.io/projected/7fe91845-9a01-43ed-a1c9-bab12f0498d7-kube-api-access-5ggwr\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:33.122937 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:02:33.122903 2554 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-ipp-b-c6e95b: failed to sync configmap cache: timed out waiting for the condition Jul 24 23:02:33.122937 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:02:33.122919 2554 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-ipp-b-c6e95b: failed to sync configmap cache: timed out waiting for the condition Jul 24 23:02:33.123404 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:02:33.123009 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume podName:7fe91845-9a01-43ed-a1c9-bab12f0498d7 nodeName:}" failed. No retries permitted until 2026-07-24 23:02:33.622990955 +0000 UTC m=+2851.009808978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume") pod "payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" (UID: "7fe91845-9a01-43ed-a1c9-bab12f0498d7") : failed to sync configmap cache: timed out waiting for the condition Jul 24 23:02:33.123404 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:02:33.123026 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume podName:ec4cda66-cfcc-42d7-91ae-ea9958214df5 nodeName:}" failed. No retries permitted until 2026-07-24 23:02:33.623018956 +0000 UTC m=+2851.009836979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume") pod "payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" (UID: "ec4cda66-cfcc-42d7-91ae-ea9958214df5") : failed to sync configmap cache: timed out waiting for the condition Jul 24 23:02:33.383514 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.383439 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c6e95b\"" Jul 24 23:02:33.641909 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.641867 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:33.642120 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.641917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:33.642885 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.642855 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:33.643610 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.643585 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:33.809310 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.809281 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:33.817334 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.817310 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:33.956723 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:33.956684 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv"] Jul 24 23:02:33.958937 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:02:33.958911 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4cda66_cfcc_42d7_91ae_ea9958214df5.slice/crio-cee5af113af9bcb7f71d44d4b88933b5e65fe936fb9b2817c3108abac409cb0c WatchSource:0}: Error finding container cee5af113af9bcb7f71d44d4b88933b5e65fe936fb9b2817c3108abac409cb0c: Status 404 returned error can't find the container with id cee5af113af9bcb7f71d44d4b88933b5e65fe936fb9b2817c3108abac409cb0c Jul 24 23:02:34.191644 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.191614 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx"] Jul 24 23:02:34.192911 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:02:34.192881 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe91845_9a01_43ed_a1c9_bab12f0498d7.slice/crio-93a2439e0659d299be065de14aad475ec140d4a34618c11a287248db69a1a929 WatchSource:0}: Error finding container 93a2439e0659d299be065de14aad475ec140d4a34618c11a287248db69a1a929: Status 404 returned error can't find the container with id 93a2439e0659d299be065de14aad475ec140d4a34618c11a287248db69a1a929 Jul 24 23:02:34.817982 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.817942 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" event={"ID":"7fe91845-9a01-43ed-a1c9-bab12f0498d7","Type":"ContainerStarted","Data":"756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d"} Jul 24 23:02:34.818184 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.818010 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" event={"ID":"7fe91845-9a01-43ed-a1c9-bab12f0498d7","Type":"ContainerStarted","Data":"93a2439e0659d299be065de14aad475ec140d4a34618c11a287248db69a1a929"} Jul 24 23:02:34.818184 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.818139 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:34.819547 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.819514 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" event={"ID":"ec4cda66-cfcc-42d7-91ae-ea9958214df5","Type":"ContainerStarted","Data":"c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089"} Jul 24 23:02:34.819655 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.819566 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:34.819655 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.819583 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" event={"ID":"ec4cda66-cfcc-42d7-91ae-ea9958214df5","Type":"ContainerStarted","Data":"cee5af113af9bcb7f71d44d4b88933b5e65fe936fb9b2817c3108abac409cb0c"} Jul 24 23:02:34.834712 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.834660 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" podStartSLOduration=3.834643098 podStartE2EDuration="3.834643098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:34.833815316 +0000 UTC m=+2852.220633361" watchObservedRunningTime="2026-07-24 23:02:34.834643098 +0000 UTC m=+2852.221461143" Jul 24 23:02:34.850863 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:34.850811 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" podStartSLOduration=3.850797264 podStartE2EDuration="3.850797264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:02:34.848904306 +0000 UTC m=+2852.235722375" watchObservedRunningTime="2026-07-24 23:02:34.850797264 +0000 UTC m=+2852.237615339" Jul 24 23:02:45.826647 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:45.826613 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:02:45.827109 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:45.826679 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:02:52.559658 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:52.559612 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cbc94_19a1f037-a1ee-4488-968c-ff384c53287d/payload-processing/0.log" Jul 24 23:02:52.679078 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:02:52.679045 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv_ec4cda66-cfcc-42d7-91ae-ea9958214df5/payload-processing/0.log" Jul 24 23:03:34.186215 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:34.186181 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm_723b83ef-3f72-4ae0-a8dc-6027cc515325/payload-processing/0.log" Jul 24 23:03:34.308393 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:34.308356 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cbc94_19a1f037-a1ee-4488-968c-ff384c53287d/payload-processing/0.log" Jul 24 23:03:34.425650 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:34.425622 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv_ec4cda66-cfcc-42d7-91ae-ea9958214df5/payload-processing/0.log" Jul 24 23:03:39.083889 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.083824 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2"] Jul 24 23:03:39.087880 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.087851 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.090433 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.090400 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-34f01f\"" Jul 24 23:03:39.090781 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.090760 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-34f01f-dockercfg-cqmjl\"" Jul 24 23:03:39.094552 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.094525 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr"] Jul 24 23:03:39.098654 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.098628 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2"] Jul 24 23:03:39.098784 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.098745 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.110737 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.110688 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr"] Jul 24 23:03:39.223465 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.223421 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbrnp\" (UniqueName: \"kubernetes.io/projected/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-kube-api-access-cbrnp\") pod \"payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.223668 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.223583 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.223668 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.223615 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6442b09-fb15-464c-bd46-07f0d6408c4e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.223668 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.223638 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffwcj\" (UniqueName: \"kubernetes.io/projected/e6442b09-fb15-464c-bd46-07f0d6408c4e-kube-api-access-ffwcj\") pod \"payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.325024 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.324983 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.325024 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.325023 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6442b09-fb15-464c-bd46-07f0d6408c4e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.325335 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.325043 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffwcj\" (UniqueName: \"kubernetes.io/projected/e6442b09-fb15-464c-bd46-07f0d6408c4e-kube-api-access-ffwcj\") pod \"payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.325335 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.325081 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbrnp\" (UniqueName: \"kubernetes.io/projected/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-kube-api-access-cbrnp\") pod \"payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.325850 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.325821 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.325943 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.325831 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6442b09-fb15-464c-bd46-07f0d6408c4e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.334327 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.334254 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffwcj\" (UniqueName: \"kubernetes.io/projected/e6442b09-fb15-464c-bd46-07f0d6408c4e-kube-api-access-ffwcj\") pod \"payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.334801 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.334774 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbrnp\" (UniqueName: \"kubernetes.io/projected/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-kube-api-access-cbrnp\") pod \"payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.400739 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.400683 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:39.412287 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.412256 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:39.569344 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.569291 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2"] Jul 24 23:03:39.572639 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:03:39.572597 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6442b09_fb15_464c_bd46_07f0d6408c4e.slice/crio-2d2d143d3d26e186fe82179c6ec5bf99968e96b51ed2d72d4a84ac38a242d228 WatchSource:0}: Error finding container 2d2d143d3d26e186fe82179c6ec5bf99968e96b51ed2d72d4a84ac38a242d228: Status 404 returned error can't find the container with id 2d2d143d3d26e186fe82179c6ec5bf99968e96b51ed2d72d4a84ac38a242d228 Jul 24 23:03:39.598711 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:39.598669 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr"] Jul 24 23:03:39.614446 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:03:39.614409 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a50328f_a68b_4d7f_ab3b_abf07eaf102b.slice/crio-14e6c142d23983b63345462d629d792278a32f8b6bdb592d6c4b14c642c72ba3 WatchSource:0}: Error finding container 14e6c142d23983b63345462d629d792278a32f8b6bdb592d6c4b14c642c72ba3: Status 404 returned error can't find the container with id 14e6c142d23983b63345462d629d792278a32f8b6bdb592d6c4b14c642c72ba3 Jul 24 23:03:40.184178 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.184109 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" event={"ID":"8a50328f-a68b-4d7f-ab3b-abf07eaf102b","Type":"ContainerStarted","Data":"82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25"} Jul 24 23:03:40.184663 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.184197 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" event={"ID":"8a50328f-a68b-4d7f-ab3b-abf07eaf102b","Type":"ContainerStarted","Data":"14e6c142d23983b63345462d629d792278a32f8b6bdb592d6c4b14c642c72ba3"} Jul 24 23:03:40.184663 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.184256 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:40.185946 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.185918 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" event={"ID":"e6442b09-fb15-464c-bd46-07f0d6408c4e","Type":"ContainerStarted","Data":"1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588"} Jul 24 23:03:40.186086 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.185963 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" event={"ID":"e6442b09-fb15-464c-bd46-07f0d6408c4e","Type":"ContainerStarted","Data":"2d2d143d3d26e186fe82179c6ec5bf99968e96b51ed2d72d4a84ac38a242d228"} Jul 24 23:03:40.186086 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.186004 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:03:40.201259 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.201199 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" podStartSLOduration=1.20118087 podStartE2EDuration="1.20118087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:03:40.198940329 +0000 UTC m=+2917.585758375" watchObservedRunningTime="2026-07-24 23:03:40.20118087 +0000 UTC m=+2917.587998913" Jul 24 23:03:40.214016 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:40.213962 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" podStartSLOduration=1.213946355 podStartE2EDuration="1.213946355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 23:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:03:40.213828778 +0000 UTC m=+2917.600646846" watchObservedRunningTime="2026-07-24 23:03:40.213946355 +0000 UTC m=+2917.600764405" Jul 24 23:03:51.194056 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:51.194021 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:03:51.194589 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:03:51.194093 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:04:04.218905 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.218864 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2"] Jul 24 23:04:04.219506 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.219131 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" podUID="e6442b09-fb15-464c-bd46-07f0d6408c4e" containerName="payload-processing" containerID="cri-o://1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588" gracePeriod=30 Jul 24 23:04:04.224939 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.224913 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr"] Jul 24 23:04:04.225373 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.225318 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" podUID="8a50328f-a68b-4d7f-ab3b-abf07eaf102b" containerName="payload-pre-processing" containerID="cri-o://82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25" gracePeriod=30 Jul 24 23:04:04.499700 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.499669 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:04:04.524551 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.524525 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:04:04.568323 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568292 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-plugins-config-volume\") pod \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " Jul 24 23:04:04.568465 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568343 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbrnp\" (UniqueName: \"kubernetes.io/projected/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-kube-api-access-cbrnp\") pod \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\" (UID: \"8a50328f-a68b-4d7f-ab3b-abf07eaf102b\") " Jul 24 23:04:04.568465 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568375 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6442b09-fb15-464c-bd46-07f0d6408c4e-plugins-config-volume\") pod \"e6442b09-fb15-464c-bd46-07f0d6408c4e\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " Jul 24 23:04:04.568465 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568414 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffwcj\" (UniqueName: \"kubernetes.io/projected/e6442b09-fb15-464c-bd46-07f0d6408c4e-kube-api-access-ffwcj\") pod \"e6442b09-fb15-464c-bd46-07f0d6408c4e\" (UID: \"e6442b09-fb15-464c-bd46-07f0d6408c4e\") " Jul 24 23:04:04.568762 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568731 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a50328f-a68b-4d7f-ab3b-abf07eaf102b" (UID: "8a50328f-a68b-4d7f-ab3b-abf07eaf102b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:04:04.568896 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568788 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6442b09-fb15-464c-bd46-07f0d6408c4e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6442b09-fb15-464c-bd46-07f0d6408c4e" (UID: "e6442b09-fb15-464c-bd46-07f0d6408c4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:04:04.568896 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.568808 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:04.570584 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.570561 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-kube-api-access-cbrnp" (OuterVolumeSpecName: "kube-api-access-cbrnp") pod "8a50328f-a68b-4d7f-ab3b-abf07eaf102b" (UID: "8a50328f-a68b-4d7f-ab3b-abf07eaf102b"). InnerVolumeSpecName "kube-api-access-cbrnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:04:04.570584 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.570568 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6442b09-fb15-464c-bd46-07f0d6408c4e-kube-api-access-ffwcj" (OuterVolumeSpecName: "kube-api-access-ffwcj") pod "e6442b09-fb15-464c-bd46-07f0d6408c4e" (UID: "e6442b09-fb15-464c-bd46-07f0d6408c4e"). InnerVolumeSpecName "kube-api-access-ffwcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:04:04.669958 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.669932 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6442b09-fb15-464c-bd46-07f0d6408c4e-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:04.669958 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.669958 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffwcj\" (UniqueName: \"kubernetes.io/projected/e6442b09-fb15-464c-bd46-07f0d6408c4e-kube-api-access-ffwcj\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:04.670135 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:04.669966 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbrnp\" (UniqueName: \"kubernetes.io/projected/8a50328f-a68b-4d7f-ab3b-abf07eaf102b-kube-api-access-cbrnp\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:05.328660 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.328629 2554 generic.go:356] "Generic (PLEG): container finished" podID="8a50328f-a68b-4d7f-ab3b-abf07eaf102b" containerID="82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25" exitCode=0 Jul 24 23:04:05.329103 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.328695 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" Jul 24 23:04:05.329103 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.328714 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" event={"ID":"8a50328f-a68b-4d7f-ab3b-abf07eaf102b","Type":"ContainerDied","Data":"82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25"} Jul 24 23:04:05.329103 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.328806 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr" event={"ID":"8a50328f-a68b-4d7f-ab3b-abf07eaf102b","Type":"ContainerDied","Data":"14e6c142d23983b63345462d629d792278a32f8b6bdb592d6c4b14c642c72ba3"} Jul 24 23:04:05.329103 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.328841 2554 scope.go:117] "RemoveContainer" containerID="82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25" Jul 24 23:04:05.330339 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.330214 2554 generic.go:356] "Generic (PLEG): container finished" podID="e6442b09-fb15-464c-bd46-07f0d6408c4e" containerID="1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588" exitCode=0 Jul 24 23:04:05.330339 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.330273 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" Jul 24 23:04:05.330339 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.330310 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" event={"ID":"e6442b09-fb15-464c-bd46-07f0d6408c4e","Type":"ContainerDied","Data":"1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588"} Jul 24 23:04:05.330490 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.330359 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2" event={"ID":"e6442b09-fb15-464c-bd46-07f0d6408c4e","Type":"ContainerDied","Data":"2d2d143d3d26e186fe82179c6ec5bf99968e96b51ed2d72d4a84ac38a242d228"} Jul 24 23:04:05.338635 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.338620 2554 scope.go:117] "RemoveContainer" containerID="82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25" Jul 24 23:04:05.338865 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:04:05.338844 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25\": container with ID starting with 82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25 not found: ID does not exist" containerID="82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25" Jul 24 23:04:05.338954 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.338870 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25"} err="failed to get container status \"82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25\": rpc error: code = NotFound desc = could not find container \"82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25\": container with ID starting with 82863bee6e2710ac095b909b33a9dca4fb7d43932630b03e6b1dbae9f4527f25 not found: ID does not exist" Jul 24 23:04:05.338954 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.338886 2554 scope.go:117] "RemoveContainer" containerID="1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588" Jul 24 23:04:05.346709 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.346676 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr"] Jul 24 23:04:05.348294 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.348276 2554 scope.go:117] "RemoveContainer" containerID="1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588" Jul 24 23:04:05.348545 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:04:05.348527 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588\": container with ID starting with 1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588 not found: ID does not exist" containerID="1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588" Jul 24 23:04:05.348613 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.348556 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588"} err="failed to get container status \"1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588\": rpc error: code = NotFound desc = could not find container \"1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588\": container with ID starting with 1ddf81ff3f6152a9bd762130774fe682247585dbe0d3e84f6bc6c375c1a8a588 not found: ID does not exist" Jul 24 23:04:05.350595 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.350575 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-34f01f-6d9dc64684-r8dgr"] Jul 24 23:04:05.360189 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.360168 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2"] Jul 24 23:04:05.365271 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:05.365250 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34f01f-f8684c588-vprc2"] Jul 24 23:04:07.176354 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:07.176322 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a50328f-a68b-4d7f-ab3b-abf07eaf102b" path="/var/lib/kubelet/pods/8a50328f-a68b-4d7f-ab3b-abf07eaf102b/volumes" Jul 24 23:04:07.176716 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:07.176648 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6442b09-fb15-464c-bd46-07f0d6408c4e" path="/var/lib/kubelet/pods/e6442b09-fb15-464c-bd46-07f0d6408c4e/volumes" Jul 24 23:04:34.032187 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.032134 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm"] Jul 24 23:04:34.032626 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.032459 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" podUID="723b83ef-3f72-4ae0-a8dc-6027cc515325" containerName="payload-processing" containerID="cri-o://89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5" gracePeriod=30 Jul 24 23:04:34.040121 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.040091 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88"] Jul 24 23:04:34.040414 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.040369 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" podUID="1cfdc8ce-4fca-4318-af68-cba2f86e3932" containerName="payload-pre-processing" containerID="cri-o://61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5" gracePeriod=30 Jul 24 23:04:34.332469 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.332444 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:04:34.335207 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.335190 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:04:34.427564 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.427533 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjfmz\" (UniqueName: \"kubernetes.io/projected/1cfdc8ce-4fca-4318-af68-cba2f86e3932-kube-api-access-kjfmz\") pod \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " Jul 24 23:04:34.427734 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.427621 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cfdc8ce-4fca-4318-af68-cba2f86e3932-plugins-config-volume\") pod \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\" (UID: \"1cfdc8ce-4fca-4318-af68-cba2f86e3932\") " Jul 24 23:04:34.427734 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.427646 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzqxf\" (UniqueName: \"kubernetes.io/projected/723b83ef-3f72-4ae0-a8dc-6027cc515325-kube-api-access-xzqxf\") pod \"723b83ef-3f72-4ae0-a8dc-6027cc515325\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " Jul 24 23:04:34.427734 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.427673 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723b83ef-3f72-4ae0-a8dc-6027cc515325-plugins-config-volume\") pod \"723b83ef-3f72-4ae0-a8dc-6027cc515325\" (UID: \"723b83ef-3f72-4ae0-a8dc-6027cc515325\") " Jul 24 23:04:34.428073 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.428049 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cfdc8ce-4fca-4318-af68-cba2f86e3932-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1cfdc8ce-4fca-4318-af68-cba2f86e3932" (UID: "1cfdc8ce-4fca-4318-af68-cba2f86e3932"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:04:34.428254 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.428085 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723b83ef-3f72-4ae0-a8dc-6027cc515325-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "723b83ef-3f72-4ae0-a8dc-6027cc515325" (UID: "723b83ef-3f72-4ae0-a8dc-6027cc515325"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:04:34.429918 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.429889 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cfdc8ce-4fca-4318-af68-cba2f86e3932-kube-api-access-kjfmz" (OuterVolumeSpecName: "kube-api-access-kjfmz") pod "1cfdc8ce-4fca-4318-af68-cba2f86e3932" (UID: "1cfdc8ce-4fca-4318-af68-cba2f86e3932"). InnerVolumeSpecName "kube-api-access-kjfmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:04:34.430009 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.429925 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723b83ef-3f72-4ae0-a8dc-6027cc515325-kube-api-access-xzqxf" (OuterVolumeSpecName: "kube-api-access-xzqxf") pod "723b83ef-3f72-4ae0-a8dc-6027cc515325" (UID: "723b83ef-3f72-4ae0-a8dc-6027cc515325"). InnerVolumeSpecName "kube-api-access-xzqxf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:04:34.488067 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.488034 2554 generic.go:356] "Generic (PLEG): container finished" podID="1cfdc8ce-4fca-4318-af68-cba2f86e3932" containerID="61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5" exitCode=0 Jul 24 23:04:34.488250 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.488085 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" Jul 24 23:04:34.488250 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.488126 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" event={"ID":"1cfdc8ce-4fca-4318-af68-cba2f86e3932","Type":"ContainerDied","Data":"61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5"} Jul 24 23:04:34.488250 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.488217 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88" event={"ID":"1cfdc8ce-4fca-4318-af68-cba2f86e3932","Type":"ContainerDied","Data":"fd828641e9dd06b55a2e2a52bc77e882b6effae976e7cc2cfd43efc467e5351c"} Jul 24 23:04:34.488250 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.488243 2554 scope.go:117] "RemoveContainer" containerID="61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5" Jul 24 23:04:34.489624 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.489599 2554 generic.go:356] "Generic (PLEG): container finished" podID="723b83ef-3f72-4ae0-a8dc-6027cc515325" containerID="89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5" exitCode=0 Jul 24 23:04:34.489763 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.489644 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" Jul 24 23:04:34.489763 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.489691 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" event={"ID":"723b83ef-3f72-4ae0-a8dc-6027cc515325","Type":"ContainerDied","Data":"89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5"} Jul 24 23:04:34.489763 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.489730 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm" event={"ID":"723b83ef-3f72-4ae0-a8dc-6027cc515325","Type":"ContainerDied","Data":"9816e4ff27f6eb57db7dabfa439635c8efbe70f3fcc95c318795d8d0605f66a6"} Jul 24 23:04:34.498886 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.498869 2554 scope.go:117] "RemoveContainer" containerID="61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5" Jul 24 23:04:34.499216 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:04:34.499124 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5\": container with ID starting with 61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5 not found: ID does not exist" containerID="61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5" Jul 24 23:04:34.499216 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.499172 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5"} err="failed to get container status \"61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5\": rpc error: code = NotFound desc = could not find container \"61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5\": container with ID starting with 61105e5b6ea5b507a7398cb72dbb01fc43f999b7c401ca5e5298413e7c0ef7e5 not found: ID does not exist" Jul 24 23:04:34.499216 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.499195 2554 scope.go:117] "RemoveContainer" containerID="89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5" Jul 24 23:04:34.508474 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.508453 2554 scope.go:117] "RemoveContainer" containerID="89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5" Jul 24 23:04:34.508820 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:04:34.508797 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5\": container with ID starting with 89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5 not found: ID does not exist" containerID="89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5" Jul 24 23:04:34.508899 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.508828 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5"} err="failed to get container status \"89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5\": rpc error: code = NotFound desc = could not find container \"89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5\": container with ID starting with 89c597ccda5ccc44b0d9d5929907b6e5fa2d7ecbdd5fb13328ef14ffd541cce5 not found: ID does not exist" Jul 24 23:04:34.511590 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.511570 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88"] Jul 24 23:04:34.514887 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.514865 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e3f0a9-7d4d78f456-wtj88"] Jul 24 23:04:34.524594 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.524571 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm"] Jul 24 23:04:34.526953 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.526931 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-e3f0a9-5b4956c5f8-5pbpm"] Jul 24 23:04:34.529034 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.529017 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjfmz\" (UniqueName: \"kubernetes.io/projected/1cfdc8ce-4fca-4318-af68-cba2f86e3932-kube-api-access-kjfmz\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:34.529094 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.529036 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cfdc8ce-4fca-4318-af68-cba2f86e3932-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:34.529094 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.529047 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzqxf\" (UniqueName: \"kubernetes.io/projected/723b83ef-3f72-4ae0-a8dc-6027cc515325-kube-api-access-xzqxf\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:34.529094 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:34.529055 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/723b83ef-3f72-4ae0-a8dc-6027cc515325-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:04:35.176437 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:35.176405 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cfdc8ce-4fca-4318-af68-cba2f86e3932" path="/var/lib/kubelet/pods/1cfdc8ce-4fca-4318-af68-cba2f86e3932/volumes" Jul 24 23:04:35.176805 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:04:35.176754 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723b83ef-3f72-4ae0-a8dc-6027cc515325" path="/var/lib/kubelet/pods/723b83ef-3f72-4ae0-a8dc-6027cc515325/volumes" Jul 24 23:05:38.137525 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.137482 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv"] Jul 24 23:05:38.138127 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.137716 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" podUID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" containerName="payload-processing" containerID="cri-o://c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089" gracePeriod=30 Jul 24 23:05:38.143085 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.143050 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx"] Jul 24 23:05:38.143358 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.143295 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" podUID="7fe91845-9a01-43ed-a1c9-bab12f0498d7" containerName="payload-pre-processing" containerID="cri-o://756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d" gracePeriod=30 Jul 24 23:05:38.438563 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.438532 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:05:38.441396 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.441378 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:05:38.577499 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.577462 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ggwr\" (UniqueName: \"kubernetes.io/projected/7fe91845-9a01-43ed-a1c9-bab12f0498d7-kube-api-access-5ggwr\") pod \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " Jul 24 23:05:38.577700 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.577529 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92s5p\" (UniqueName: \"kubernetes.io/projected/ec4cda66-cfcc-42d7-91ae-ea9958214df5-kube-api-access-92s5p\") pod \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " Jul 24 23:05:38.577700 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.577582 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume\") pod \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\" (UID: \"7fe91845-9a01-43ed-a1c9-bab12f0498d7\") " Jul 24 23:05:38.577700 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.577601 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume\") pod \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\" (UID: \"ec4cda66-cfcc-42d7-91ae-ea9958214df5\") " Jul 24 23:05:38.578019 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.577993 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7fe91845-9a01-43ed-a1c9-bab12f0498d7" (UID: "7fe91845-9a01-43ed-a1c9-bab12f0498d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:05:38.578107 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.578028 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec4cda66-cfcc-42d7-91ae-ea9958214df5" (UID: "ec4cda66-cfcc-42d7-91ae-ea9958214df5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:05:38.579799 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.579777 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4cda66-cfcc-42d7-91ae-ea9958214df5-kube-api-access-92s5p" (OuterVolumeSpecName: "kube-api-access-92s5p") pod "ec4cda66-cfcc-42d7-91ae-ea9958214df5" (UID: "ec4cda66-cfcc-42d7-91ae-ea9958214df5"). InnerVolumeSpecName "kube-api-access-92s5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:05:38.579842 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.579805 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fe91845-9a01-43ed-a1c9-bab12f0498d7-kube-api-access-5ggwr" (OuterVolumeSpecName: "kube-api-access-5ggwr") pod "7fe91845-9a01-43ed-a1c9-bab12f0498d7" (UID: "7fe91845-9a01-43ed-a1c9-bab12f0498d7"). InnerVolumeSpecName "kube-api-access-5ggwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:05:38.678911 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.678825 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92s5p\" (UniqueName: \"kubernetes.io/projected/ec4cda66-cfcc-42d7-91ae-ea9958214df5-kube-api-access-92s5p\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:05:38.678911 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.678855 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fe91845-9a01-43ed-a1c9-bab12f0498d7-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:05:38.678911 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.678864 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4cda66-cfcc-42d7-91ae-ea9958214df5-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:05:38.678911 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.678874 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ggwr\" (UniqueName: \"kubernetes.io/projected/7fe91845-9a01-43ed-a1c9-bab12f0498d7-kube-api-access-5ggwr\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:05:38.837031 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.836995 2554 generic.go:356] "Generic (PLEG): container finished" podID="7fe91845-9a01-43ed-a1c9-bab12f0498d7" containerID="756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d" exitCode=0 Jul 24 23:05:38.837243 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.837038 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" Jul 24 23:05:38.837243 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.837080 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" event={"ID":"7fe91845-9a01-43ed-a1c9-bab12f0498d7","Type":"ContainerDied","Data":"756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d"} Jul 24 23:05:38.837243 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.837143 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx" event={"ID":"7fe91845-9a01-43ed-a1c9-bab12f0498d7","Type":"ContainerDied","Data":"93a2439e0659d299be065de14aad475ec140d4a34618c11a287248db69a1a929"} Jul 24 23:05:38.837243 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.837183 2554 scope.go:117] "RemoveContainer" containerID="756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d" Jul 24 23:05:38.838370 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.838350 2554 generic.go:356] "Generic (PLEG): container finished" podID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" containerID="c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089" exitCode=0 Jul 24 23:05:38.838450 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.838390 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" Jul 24 23:05:38.838450 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.838427 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" event={"ID":"ec4cda66-cfcc-42d7-91ae-ea9958214df5","Type":"ContainerDied","Data":"c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089"} Jul 24 23:05:38.838552 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.838467 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv" event={"ID":"ec4cda66-cfcc-42d7-91ae-ea9958214df5","Type":"ContainerDied","Data":"cee5af113af9bcb7f71d44d4b88933b5e65fe936fb9b2817c3108abac409cb0c"} Jul 24 23:05:38.847324 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.847301 2554 scope.go:117] "RemoveContainer" containerID="756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d" Jul 24 23:05:38.847672 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:05:38.847644 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d\": container with ID starting with 756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d not found: ID does not exist" containerID="756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d" Jul 24 23:05:38.847759 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.847681 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d"} err="failed to get container status \"756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d\": rpc error: code = NotFound desc = could not find container \"756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d\": container with ID starting with 756bd2a42fc7a49c1b37ec53e46409589c07e65bad64d71f3496920370b5530d not found: ID does not exist" Jul 24 23:05:38.847759 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.847697 2554 scope.go:117] "RemoveContainer" containerID="c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089" Jul 24 23:05:38.856509 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.856491 2554 scope.go:117] "RemoveContainer" containerID="c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089" Jul 24 23:05:38.856779 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:05:38.856756 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089\": container with ID starting with c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089 not found: ID does not exist" containerID="c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089" Jul 24 23:05:38.856826 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.856787 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089"} err="failed to get container status \"c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089\": rpc error: code = NotFound desc = could not find container \"c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089\": container with ID starting with c6729ec5de79b0065d8a80179c2f0b84d353e355d5940f0b42f2c73de5de0089 not found: ID does not exist" Jul 24 23:05:38.863077 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.862646 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx"] Jul 24 23:05:38.867982 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.867947 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c6e95b-6549469f7b-9l8kx"] Jul 24 23:05:38.875847 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.875825 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv"] Jul 24 23:05:38.880201 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:38.880177 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c6e95b-7599796564-dshgv"] Jul 24 23:05:39.175900 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:39.175870 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fe91845-9a01-43ed-a1c9-bab12f0498d7" path="/var/lib/kubelet/pods/7fe91845-9a01-43ed-a1c9-bab12f0498d7/volumes" Jul 24 23:05:39.176275 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:39.176242 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" path="/var/lib/kubelet/pods/ec4cda66-cfcc-42d7-91ae-ea9958214df5/volumes" Jul 24 23:05:59.569201 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.569097 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc"] Jul 24 23:05:59.571182 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.571140 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-499dd1-29748900-7v8fc"] Jul 24 23:05:59.592606 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.592579 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx"] Jul 24 23:05:59.592868 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.592817 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" podUID="ed0bee41-0a14-4b10-a36d-8c9ef708dd22" containerName="payload-processing" containerID="cri-o://a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092" gracePeriod=30 Jul 24 23:05:59.602708 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.602682 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k"] Jul 24 23:05:59.602891 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.602870 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" podUID="77fc5a60-3070-4904-a9c1-c8a4b6d9d123" containerName="payload-pre-processing" containerID="cri-o://b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0" gracePeriod=30 Jul 24 23:05:59.860329 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.860302 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 23:05:59.897384 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.897360 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 23:05:59.942190 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.942139 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-plugins-config-volume\") pod \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " Jul 24 23:05:59.942190 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.942197 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-plugins-config-volume\") pod \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " Jul 24 23:05:59.942407 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.942229 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kp77\" (UniqueName: \"kubernetes.io/projected/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-kube-api-access-8kp77\") pod \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\" (UID: \"ed0bee41-0a14-4b10-a36d-8c9ef708dd22\") " Jul 24 23:05:59.942407 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.942291 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vsw2n\" (UniqueName: \"kubernetes.io/projected/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-kube-api-access-vsw2n\") pod \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\" (UID: \"77fc5a60-3070-4904-a9c1-c8a4b6d9d123\") " Jul 24 23:05:59.942585 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.942555 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "77fc5a60-3070-4904-a9c1-c8a4b6d9d123" (UID: "77fc5a60-3070-4904-a9c1-c8a4b6d9d123"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:05:59.942658 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.942562 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed0bee41-0a14-4b10-a36d-8c9ef708dd22" (UID: "ed0bee41-0a14-4b10-a36d-8c9ef708dd22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:05:59.944468 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.944444 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-kube-api-access-8kp77" (OuterVolumeSpecName: "kube-api-access-8kp77") pod "ed0bee41-0a14-4b10-a36d-8c9ef708dd22" (UID: "ed0bee41-0a14-4b10-a36d-8c9ef708dd22"). InnerVolumeSpecName "kube-api-access-8kp77". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:05:59.944468 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.944455 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-kube-api-access-vsw2n" (OuterVolumeSpecName: "kube-api-access-vsw2n") pod "77fc5a60-3070-4904-a9c1-c8a4b6d9d123" (UID: "77fc5a60-3070-4904-a9c1-c8a4b6d9d123"). InnerVolumeSpecName "kube-api-access-vsw2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:05:59.952278 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.952249 2554 generic.go:356] "Generic (PLEG): container finished" podID="ed0bee41-0a14-4b10-a36d-8c9ef708dd22" containerID="a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092" exitCode=0 Jul 24 23:05:59.952388 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.952286 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" Jul 24 23:05:59.952388 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.952329 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" event={"ID":"ed0bee41-0a14-4b10-a36d-8c9ef708dd22","Type":"ContainerDied","Data":"a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092"} Jul 24 23:05:59.952388 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.952377 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx" event={"ID":"ed0bee41-0a14-4b10-a36d-8c9ef708dd22","Type":"ContainerDied","Data":"91ca37ebc1dc1d639fa8c749110a1270493b53c9025ba70255479ef27e0bd6b4"} Jul 24 23:05:59.952513 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.952409 2554 scope.go:117] "RemoveContainer" containerID="a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092" Jul 24 23:05:59.953593 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.953560 2554 generic.go:356] "Generic (PLEG): container finished" podID="77fc5a60-3070-4904-a9c1-c8a4b6d9d123" containerID="b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0" exitCode=0 Jul 24 23:05:59.953696 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.953596 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" Jul 24 23:05:59.953696 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.953647 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" event={"ID":"77fc5a60-3070-4904-a9c1-c8a4b6d9d123","Type":"ContainerDied","Data":"b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0"} Jul 24 23:05:59.953696 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.953685 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k" event={"ID":"77fc5a60-3070-4904-a9c1-c8a4b6d9d123","Type":"ContainerDied","Data":"9e541cecb5673dde72927b802fddc039736e1dbf5deddd75de05f62a5efbcbb4"} Jul 24 23:05:59.960982 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.960966 2554 scope.go:117] "RemoveContainer" containerID="a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092" Jul 24 23:05:59.961316 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:05:59.961290 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092\": container with ID starting with a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092 not found: ID does not exist" containerID="a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092" Jul 24 23:05:59.961397 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.961327 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092"} err="failed to get container status \"a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092\": rpc error: code = NotFound desc = could not find container \"a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092\": container with ID starting with a56c293eb27a45ea8a0bdf776dad882ac7ff00000fc0e402644f89db12157092 not found: ID does not exist" Jul 24 23:05:59.961397 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.961349 2554 scope.go:117] "RemoveContainer" containerID="b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0" Jul 24 23:05:59.969630 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.969615 2554 scope.go:117] "RemoveContainer" containerID="b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0" Jul 24 23:05:59.969885 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:05:59.969866 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0\": container with ID starting with b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0 not found: ID does not exist" containerID="b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0" Jul 24 23:05:59.969931 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.969892 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0"} err="failed to get container status \"b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0\": rpc error: code = NotFound desc = could not find container \"b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0\": container with ID starting with b53bbbc932fb71781a89b04c02046a5ef2b63a8b56e016a4a2c325bedac172c0 not found: ID does not exist" Jul 24 23:05:59.976781 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.976760 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx"] Jul 24 23:05:59.980326 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.980307 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-499dd1-794dc545d6-fqrtx"] Jul 24 23:05:59.990541 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.990521 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k"] Jul 24 23:05:59.992470 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:05:59.992449 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-499dd1-7c7bdd9bbd-tsm5k"] Jul 24 23:06:00.043409 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:00.043387 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsw2n\" (UniqueName: \"kubernetes.io/projected/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-kube-api-access-vsw2n\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:00.043409 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:00.043408 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77fc5a60-3070-4904-a9c1-c8a4b6d9d123-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:00.043546 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:00.043419 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:00.043546 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:00.043428 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kp77\" (UniqueName: \"kubernetes.io/projected/ed0bee41-0a14-4b10-a36d-8c9ef708dd22-kube-api-access-8kp77\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:01.176222 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:01.176188 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57710237-1f7b-45ac-aac7-a599d4eb1ecb" path="/var/lib/kubelet/pods/57710237-1f7b-45ac-aac7-a599d4eb1ecb/volumes" Jul 24 23:06:01.176610 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:01.176523 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77fc5a60-3070-4904-a9c1-c8a4b6d9d123" path="/var/lib/kubelet/pods/77fc5a60-3070-4904-a9c1-c8a4b6d9d123/volumes" Jul 24 23:06:01.176829 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:01.176816 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed0bee41-0a14-4b10-a36d-8c9ef708dd22" path="/var/lib/kubelet/pods/ed0bee41-0a14-4b10-a36d-8c9ef708dd22/volumes" Jul 24 23:06:03.548080 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:03.548051 2554 scope.go:117] "RemoveContainer" containerID="bf9ab3e25afee1523b8874423230b2fd7cf8e67c896bd2eea3b5113e76d23d56" Jul 24 23:06:10.707983 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:10.707943 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99"] Jul 24 23:06:10.708543 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:10.708218 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" containerID="cri-o://d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468" gracePeriod=30 Jul 24 23:06:15.959051 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:15.959027 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 23:06:16.038640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.038561 2554 generic.go:356] "Generic (PLEG): container finished" podID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerID="d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468" exitCode=0 Jul 24 23:06:16.038640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.038608 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" Jul 24 23:06:16.038869 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.038649 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" event={"ID":"fcb80707-e293-4a11-ab07-a5700d3abc57","Type":"ContainerDied","Data":"d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468"} Jul 24 23:06:16.038869 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.038717 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99" event={"ID":"fcb80707-e293-4a11-ab07-a5700d3abc57","Type":"ContainerDied","Data":"1921bb7e67abd1b56602a6d9e2fea733f84393c06b034e8c0ef3b4d63b2ded0f"} Jul 24 23:06:16.038869 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.038750 2554 scope.go:117] "RemoveContainer" containerID="d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468" Jul 24 23:06:16.047271 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.047252 2554 scope.go:117] "RemoveContainer" containerID="64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708" Jul 24 23:06:16.054939 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.054915 2554 scope.go:117] "RemoveContainer" containerID="d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468" Jul 24 23:06:16.055361 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:06:16.055340 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468\": container with ID starting with d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468 not found: ID does not exist" containerID="d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468" Jul 24 23:06:16.055440 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.055371 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468"} err="failed to get container status \"d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468\": rpc error: code = NotFound desc = could not find container \"d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468\": container with ID starting with d5a0fea4a430048534bfce64097f867d7340002c277fd3f8bf932c16bbd74468 not found: ID does not exist" Jul 24 23:06:16.055440 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.055388 2554 scope.go:117] "RemoveContainer" containerID="64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708" Jul 24 23:06:16.055642 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:06:16.055626 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708\": container with ID starting with 64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708 not found: ID does not exist" containerID="64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708" Jul 24 23:06:16.055688 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.055646 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708"} err="failed to get container status \"64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708\": rpc error: code = NotFound desc = could not find container \"64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708\": container with ID starting with 64b17320a5bef3c2ed51f36d9f78854bc2bba1633ff6abfbbb6f9d52e4573708 not found: ID does not exist" Jul 24 23:06:16.086408 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086382 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-token\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086536 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086411 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb80707-e293-4a11-ab07-a5700d3abc57-istiod-ca-cert\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086536 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086457 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-credential-socket\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086536 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086479 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjfg8\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-kube-api-access-vjfg8\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086536 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086509 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-data\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086743 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086571 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-certs\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086743 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086596 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-socket\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.086862 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086833 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:16.086916 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086877 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:16.086916 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086906 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcb80707-e293-4a11-ab07-a5700d3abc57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:06:16.087027 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086943 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-envoy\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.087027 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086965 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-podinfo\") pod \"fcb80707-e293-4a11-ab07-a5700d3abc57\" (UID: \"fcb80707-e293-4a11-ab07-a5700d3abc57\") " Jul 24 23:06:16.087027 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.086928 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:16.087385 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.087273 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.087385 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.087292 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.087385 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.087304 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcb80707-e293-4a11-ab07-a5700d3abc57-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.087385 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.087318 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.087599 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.087532 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-data" (OuterVolumeSpecName: "istio-data") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:16.088905 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.088878 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-kube-api-access-vjfg8" (OuterVolumeSpecName: "kube-api-access-vjfg8") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "kube-api-access-vjfg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:06:16.088991 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.088884 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-token" (OuterVolumeSpecName: "istio-token") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:06:16.089032 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.088990 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:16.089387 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.089367 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fcb80707-e293-4a11-ab07-a5700d3abc57" (UID: "fcb80707-e293-4a11-ab07-a5700d3abc57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 23:06:16.188406 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.188373 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjfg8\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-kube-api-access-vjfg8\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.188406 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.188402 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.188568 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.188416 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.188568 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.188425 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.188568 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.188431 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcb80707-e293-4a11-ab07-a5700d3abc57-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:16.362132 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.362102 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99"] Jul 24 23:06:16.367138 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:16.367115 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-499dd1-openshift-default-758c466b96-pvn99"] Jul 24 23:06:17.175620 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:17.175589 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" path="/var/lib/kubelet/pods/fcb80707-e293-4a11-ab07-a5700d3abc57/volumes" Jul 24 23:06:21.605613 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:21.605581 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4"] Jul 24 23:06:21.609924 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:21.609899 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-e15986-29748900-wccp4"] Jul 24 23:06:21.633140 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:21.633113 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg"] Jul 24 23:06:21.633380 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:21.633356 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" podUID="f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" containerName="payload-processing" containerID="cri-o://80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677" gracePeriod=30 Jul 24 23:06:21.641509 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:21.641481 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6"] Jul 24 23:06:21.896401 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:21.896373 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 23:06:22.038943 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.038902 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f62r\" (UniqueName: \"kubernetes.io/projected/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-kube-api-access-4f62r\") pod \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " Jul 24 23:06:22.039122 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.038963 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-plugins-config-volume\") pod \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\" (UID: \"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1\") " Jul 24 23:06:22.039385 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.039355 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" (UID: "f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:06:22.041184 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.041146 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-kube-api-access-4f62r" (OuterVolumeSpecName: "kube-api-access-4f62r") pod "f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" (UID: "f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1"). InnerVolumeSpecName "kube-api-access-4f62r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:06:22.069489 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.069466 2554 generic.go:356] "Generic (PLEG): container finished" podID="f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" containerID="80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677" exitCode=0 Jul 24 23:06:22.069596 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.069506 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" Jul 24 23:06:22.069596 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.069540 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" event={"ID":"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1","Type":"ContainerDied","Data":"80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677"} Jul 24 23:06:22.069596 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.069581 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg" event={"ID":"f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1","Type":"ContainerDied","Data":"bdaa039583463cabd68f74683fa025c94396f8f6bf428f95c9337e9c83aa8174"} Jul 24 23:06:22.069702 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.069602 2554 scope.go:117] "RemoveContainer" containerID="80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677" Jul 24 23:06:22.069873 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.069845 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" podUID="2912b3cb-3ec7-4159-b63c-edc7391fa06a" containerName="payload-pre-processing" containerID="cri-o://0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c" gracePeriod=30 Jul 24 23:06:22.078969 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.078945 2554 scope.go:117] "RemoveContainer" containerID="80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677" Jul 24 23:06:22.079276 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:06:22.079258 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677\": container with ID starting with 80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677 not found: ID does not exist" containerID="80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677" Jul 24 23:06:22.079352 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.079287 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677"} err="failed to get container status \"80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677\": rpc error: code = NotFound desc = could not find container \"80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677\": container with ID starting with 80a5f4012475971afa238e14e723052fab25937b4200eb7726338ec1fe1ee677 not found: ID does not exist" Jul 24 23:06:22.102105 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.102076 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg"] Jul 24 23:06:22.105042 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.105019 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-e15986-96d7d798b-prtrg"] Jul 24 23:06:22.139904 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.139879 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f62r\" (UniqueName: \"kubernetes.io/projected/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-kube-api-access-4f62r\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:22.139904 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.139901 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:22.308144 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.308120 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 23:06:22.443222 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.443128 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2912b3cb-3ec7-4159-b63c-edc7391fa06a-plugins-config-volume\") pod \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " Jul 24 23:06:22.443222 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.443191 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkvrc\" (UniqueName: \"kubernetes.io/projected/2912b3cb-3ec7-4159-b63c-edc7391fa06a-kube-api-access-tkvrc\") pod \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\" (UID: \"2912b3cb-3ec7-4159-b63c-edc7391fa06a\") " Jul 24 23:06:22.443500 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.443476 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2912b3cb-3ec7-4159-b63c-edc7391fa06a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2912b3cb-3ec7-4159-b63c-edc7391fa06a" (UID: "2912b3cb-3ec7-4159-b63c-edc7391fa06a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:06:22.445461 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.445440 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2912b3cb-3ec7-4159-b63c-edc7391fa06a-kube-api-access-tkvrc" (OuterVolumeSpecName: "kube-api-access-tkvrc") pod "2912b3cb-3ec7-4159-b63c-edc7391fa06a" (UID: "2912b3cb-3ec7-4159-b63c-edc7391fa06a"). InnerVolumeSpecName "kube-api-access-tkvrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:06:22.544095 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.544057 2554 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2912b3cb-3ec7-4159-b63c-edc7391fa06a-plugins-config-volume\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:22.544095 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:22.544088 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkvrc\" (UniqueName: \"kubernetes.io/projected/2912b3cb-3ec7-4159-b63c-edc7391fa06a-kube-api-access-tkvrc\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:23.076883 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.076846 2554 generic.go:356] "Generic (PLEG): container finished" podID="2912b3cb-3ec7-4159-b63c-edc7391fa06a" containerID="0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c" exitCode=0 Jul 24 23:06:23.076883 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.076887 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" Jul 24 23:06:23.077436 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.076900 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" event={"ID":"2912b3cb-3ec7-4159-b63c-edc7391fa06a","Type":"ContainerDied","Data":"0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c"} Jul 24 23:06:23.077436 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.076951 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6" event={"ID":"2912b3cb-3ec7-4159-b63c-edc7391fa06a","Type":"ContainerDied","Data":"c155bc918d20468cfccb6414ee965f8cae45096d58c7328f4940eda4274ec1f8"} Jul 24 23:06:23.077436 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.076993 2554 scope.go:117] "RemoveContainer" containerID="0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c" Jul 24 23:06:23.086055 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.086034 2554 scope.go:117] "RemoveContainer" containerID="0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c" Jul 24 23:06:23.086327 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:06:23.086308 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c\": container with ID starting with 0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c not found: ID does not exist" containerID="0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c" Jul 24 23:06:23.086400 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.086335 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c"} err="failed to get container status \"0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c\": rpc error: code = NotFound desc = could not find container \"0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c\": container with ID starting with 0860587bbd087519c820fffba5f08a76b317c3601b4c6a2d618bfda689d7c27c not found: ID does not exist" Jul 24 23:06:23.098111 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.098087 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6"] Jul 24 23:06:23.102024 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.101993 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-e15986-7687667dbd-dpdf6"] Jul 24 23:06:23.175659 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.175624 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2912b3cb-3ec7-4159-b63c-edc7391fa06a" path="/var/lib/kubelet/pods/2912b3cb-3ec7-4159-b63c-edc7391fa06a/volumes" Jul 24 23:06:23.175965 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.175946 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d577c762-5323-4a98-87c0-df7fcee8f7b0" path="/var/lib/kubelet/pods/d577c762-5323-4a98-87c0-df7fcee8f7b0/volumes" Jul 24 23:06:23.176319 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:23.176302 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" path="/var/lib/kubelet/pods/f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1/volumes" Jul 24 23:06:32.527499 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:32.527458 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d"] Jul 24 23:06:32.527972 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:32.527721 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" containerID="cri-o://8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f" gracePeriod=30 Jul 24 23:06:37.774438 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.774418 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 23:06:37.868741 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868664 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0035146-9042-4f88-9388-d728b78bd60f-istiod-ca-cert\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.868741 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868700 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-certs\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.868741 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868720 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-envoy\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.868741 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868736 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-istio-token\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.869050 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868760 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-socket\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.869050 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868802 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-data\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.869050 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868820 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxpdt\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-kube-api-access-lxpdt\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.869050 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868835 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0035146-9042-4f88-9388-d728b78bd60f-istio-podinfo\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.869050 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868876 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-credential-socket\") pod \"b0035146-9042-4f88-9388-d728b78bd60f\" (UID: \"b0035146-9042-4f88-9388-d728b78bd60f\") " Jul 24 23:06:37.869050 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.868976 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:37.869396 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.869193 2554 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-certs\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.869396 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.869200 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:37.869396 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.869214 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:37.869396 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.869263 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0035146-9042-4f88-9388-d728b78bd60f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 23:06:37.870097 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.870069 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:37.871245 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.871131 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-kube-api-access-lxpdt" (OuterVolumeSpecName: "kube-api-access-lxpdt") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "kube-api-access-lxpdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:06:37.871356 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.871241 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 23:06:37.871536 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.871512 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0035146-9042-4f88-9388-d728b78bd60f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 23:06:37.871722 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.871700 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0035146-9042-4f88-9388-d728b78bd60f" (UID: "b0035146-9042-4f88-9388-d728b78bd60f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 23:06:37.970429 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970401 2554 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-credential-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970429 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970425 2554 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0035146-9042-4f88-9388-d728b78bd60f-istiod-ca-cert\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970581 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970438 2554 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-envoy\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970581 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970450 2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-istio-token\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970581 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970460 2554 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-workload-socket\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970581 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970471 2554 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0035146-9042-4f88-9388-d728b78bd60f-istio-data\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970581 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970483 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxpdt\" (UniqueName: \"kubernetes.io/projected/b0035146-9042-4f88-9388-d728b78bd60f-kube-api-access-lxpdt\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:37.970581 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:37.970495 2554 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0035146-9042-4f88-9388-d728b78bd60f-istio-podinfo\") on node \"ip-10-0-143-53.ec2.internal\" DevicePath \"\"" Jul 24 23:06:38.153648 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.153613 2554 generic.go:356] "Generic (PLEG): container finished" podID="b0035146-9042-4f88-9388-d728b78bd60f" containerID="8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f" exitCode=0 Jul 24 23:06:38.153784 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.153679 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" Jul 24 23:06:38.153784 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.153694 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" event={"ID":"b0035146-9042-4f88-9388-d728b78bd60f","Type":"ContainerDied","Data":"8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f"} Jul 24 23:06:38.153873 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.153787 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d" event={"ID":"b0035146-9042-4f88-9388-d728b78bd60f","Type":"ContainerDied","Data":"35847a4ab58b61ec5433d85e186003c009e689089516d0b77d8b7abebd6b3e4e"} Jul 24 23:06:38.153873 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.153819 2554 scope.go:117] "RemoveContainer" containerID="8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f" Jul 24 23:06:38.165753 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.165727 2554 scope.go:117] "RemoveContainer" containerID="debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46" Jul 24 23:06:38.175070 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.175053 2554 scope.go:117] "RemoveContainer" containerID="8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f" Jul 24 23:06:38.175395 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:06:38.175378 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f\": container with ID starting with 8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f not found: ID does not exist" containerID="8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f" Jul 24 23:06:38.175448 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.175403 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f"} err="failed to get container status \"8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f\": rpc error: code = NotFound desc = could not find container \"8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f\": container with ID starting with 8c848294006b066c81d94d93a65a07e15b489e5e1e29c0f9b814bc4f36a2426f not found: ID does not exist" Jul 24 23:06:38.175448 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.175421 2554 scope.go:117] "RemoveContainer" containerID="debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46" Jul 24 23:06:38.175683 ip-10-0-143-53 kubenswrapper[2554]: E0724 23:06:38.175666 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46\": container with ID starting with debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46 not found: ID does not exist" containerID="debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46" Jul 24 23:06:38.175724 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.175687 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46"} err="failed to get container status \"debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46\": rpc error: code = NotFound desc = could not find container \"debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46\": container with ID starting with debd40bc39c28d22483cfe04a192470efe477ab19c14d2bd5e9f30ba8aa52c46 not found: ID does not exist" Jul 24 23:06:38.176899 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.176880 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d"] Jul 24 23:06:38.178746 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.178729 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e15986-openshift-default-5f8d668c44-7fn4d"] Jul 24 23:06:38.585060 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:38.584985 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-656df4d8c4-g286b_e3d0e7fa-1e6b-407a-b972-ceac7be0480b/manager/0.log" Jul 24 23:06:39.153535 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:39.153510 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-854fb8676-ptctb_48e11fca-1e70-41de-b8c3-5ebe02ea63a5/copy-manifests/0.log" Jul 24 23:06:39.167264 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:39.167233 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-854fb8676-ptctb_48e11fca-1e70-41de-b8c3-5ebe02ea63a5/manager/0.log" Jul 24 23:06:39.174843 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:39.174818 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0035146-9042-4f88-9388-d728b78bd60f" path="/var/lib/kubelet/pods/b0035146-9042-4f88-9388-d728b78bd60f/volumes" Jul 24 23:06:39.616583 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:39.616491 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748870-rg7hr_fa46e4e8-10d9-47df-926b-1baa8367bbac/cleanup/0.log" Jul 24 23:06:40.667349 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.667288 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2_2b0065ae-1aeb-410b-b990-520ac3ac969a/extract/0.log" Jul 24 23:06:40.673676 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.673650 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2_2b0065ae-1aeb-410b-b990-520ac3ac969a/util/0.log" Jul 24 23:06:40.680109 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.680086 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2_2b0065ae-1aeb-410b-b990-520ac3ac969a/pull/0.log" Jul 24 23:06:40.787718 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.787693 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7_121883c5-a670-4cc0-a9bb-15260b6511a8/util/0.log" Jul 24 23:06:40.793318 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.793294 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7_121883c5-a670-4cc0-a9bb-15260b6511a8/pull/0.log" Jul 24 23:06:40.798812 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.798793 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7_121883c5-a670-4cc0-a9bb-15260b6511a8/extract/0.log" Jul 24 23:06:40.903730 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.903704 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5_de5ffce6-2dbe-413e-bd33-ae9260474008/util/0.log" Jul 24 23:06:40.909840 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.909817 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5_de5ffce6-2dbe-413e-bd33-ae9260474008/pull/0.log" Jul 24 23:06:40.916273 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:40.916255 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5_de5ffce6-2dbe-413e-bd33-ae9260474008/extract/0.log" Jul 24 23:06:41.025398 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:41.025325 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d_b1309221-e4a0-403c-8fcd-a152c3fa9c75/util/0.log" Jul 24 23:06:41.032144 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:41.032119 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d_b1309221-e4a0-403c-8fcd-a152c3fa9c75/pull/0.log" Jul 24 23:06:41.038578 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:41.038559 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d_b1309221-e4a0-403c-8fcd-a152c3fa9c75/extract/0.log" Jul 24 23:06:42.295043 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:42.295007 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq_9549c3dc-c2ef-4c2e-a716-11d3338e3234/istio-proxy/0.log" Jul 24 23:06:42.410487 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:42.410457 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-phlct_68dbf04d-ba00-4c10-bfd8-2e57db942711/discovery/0.log" Jul 24 23:06:42.977734 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:42.977703 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cbc94_19a1f037-a1ee-4488-968c-ff384c53287d/payload-processing/0.log" Jul 24 23:06:43.089434 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:43.089408 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-858d756bd9-xqm8h_43c537e5-5787-4685-8933-340514153061/router/0.log" Jul 24 23:06:43.427787 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:43.427754 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xr8jp_8f823cd0-538d-42c2-8458-57c74ddd4cb4/main/0.log" Jul 24 23:06:51.884240 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:51.884190 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xvlrl_259c7f52-5c99-47cf-bd40-9aea3ea0fc6d/global-pull-secret-syncer/0.log" Jul 24 23:06:51.966889 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:51.966859 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ld27j_0edcb77e-4272-4cc6-b46e-c90755fa671a/konnectivity-agent/0.log" Jul 24 23:06:52.050994 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:52.050965 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-53.ec2.internal_b5b643551357dc02beb09e72ad5e5fed/haproxy/0.log" Jul 24 23:06:56.988333 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:56.988285 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2_2b0065ae-1aeb-410b-b990-520ac3ac969a/extract/0.log" Jul 24 23:06:57.006271 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.006241 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2_2b0065ae-1aeb-410b-b990-520ac3ac969a/util/0.log" Jul 24 23:06:57.030594 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.030574 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lkxt2_2b0065ae-1aeb-410b-b990-520ac3ac969a/pull/0.log" Jul 24 23:06:57.070699 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.070679 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7_121883c5-a670-4cc0-a9bb-15260b6511a8/extract/0.log" Jul 24 23:06:57.087277 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.087256 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7_121883c5-a670-4cc0-a9bb-15260b6511a8/util/0.log" Jul 24 23:06:57.109941 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.109921 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0djgt7_121883c5-a670-4cc0-a9bb-15260b6511a8/pull/0.log" Jul 24 23:06:57.137187 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.137144 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5_de5ffce6-2dbe-413e-bd33-ae9260474008/extract/0.log" Jul 24 23:06:57.153409 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.153384 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5_de5ffce6-2dbe-413e-bd33-ae9260474008/util/0.log" Jul 24 23:06:57.171868 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.171846 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qmdk5_de5ffce6-2dbe-413e-bd33-ae9260474008/pull/0.log" Jul 24 23:06:57.223067 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.223046 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d_b1309221-e4a0-403c-8fcd-a152c3fa9c75/extract/0.log" Jul 24 23:06:57.245376 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.245317 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d_b1309221-e4a0-403c-8fcd-a152c3fa9c75/util/0.log" Jul 24 23:06:57.263166 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:06:57.263131 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cnm2d_b1309221-e4a0-403c-8fcd-a152c3fa9c75/pull/0.log" Jul 24 23:07:00.201558 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:00.201528 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-b5ck4_2ff0d475-2d6b-4854-8748-93f562053c6f/cluster-monitoring-operator/0.log" Jul 24 23:07:00.386199 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:00.386171 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-86l6w_8d876e8d-ee84-4be9-8204-d9c8431caea5/node-exporter/0.log" Jul 24 23:07:00.401699 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:00.401674 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-86l6w_8d876e8d-ee84-4be9-8204-d9c8431caea5/kube-rbac-proxy/0.log" Jul 24 23:07:00.418659 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:00.418639 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-86l6w_8d876e8d-ee84-4be9-8204-d9c8431caea5/init-textfile/0.log" Jul 24 23:07:02.035516 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035473 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g"] Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035906 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2912b3cb-3ec7-4159-b63c-edc7391fa06a" containerName="payload-pre-processing" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035924 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="2912b3cb-3ec7-4159-b63c-edc7391fa06a" containerName="payload-pre-processing" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035944 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cfdc8ce-4fca-4318-af68-cba2f86e3932" containerName="payload-pre-processing" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035954 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cfdc8ce-4fca-4318-af68-cba2f86e3932" containerName="payload-pre-processing" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035964 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723b83ef-3f72-4ae0-a8dc-6027cc515325" containerName="payload-processing" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035972 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="723b83ef-3f72-4ae0-a8dc-6027cc515325" containerName="payload-processing" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035984 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.035991 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" Jul 24 23:07:02.036002 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036005 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036014 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036023 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036031 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036054 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0bee41-0a14-4b10-a36d-8c9ef708dd22" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036062 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0bee41-0a14-4b10-a36d-8c9ef708dd22" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036072 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036080 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036095 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77fc5a60-3070-4904-a9c1-c8a4b6d9d123" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036103 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="77fc5a60-3070-4904-a9c1-c8a4b6d9d123" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036116 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fe91845-9a01-43ed-a1c9-bab12f0498d7" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036124 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fe91845-9a01-43ed-a1c9-bab12f0498d7" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036133 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6442b09-fb15-464c-bd46-07f0d6408c4e" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036141 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6442b09-fb15-464c-bd46-07f0d6408c4e" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036173 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a50328f-a68b-4d7f-ab3b-abf07eaf102b" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036181 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a50328f-a68b-4d7f-ab3b-abf07eaf102b" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036261 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036272 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="f93c6f9f-cec2-40b9-b722-3e9e4fb47dc1" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036285 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036293 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a50328f-a68b-4d7f-ab3b-abf07eaf102b" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036302 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0035146-9042-4f88-9388-d728b78bd60f" containerName="istio-proxy" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036312 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6442b09-fb15-464c-bd46-07f0d6408c4e" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036319 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0bee41-0a14-4b10-a36d-8c9ef708dd22" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036328 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec4cda66-cfcc-42d7-91ae-ea9958214df5" containerName="payload-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036337 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fe91845-9a01-43ed-a1c9-bab12f0498d7" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036346 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cfdc8ce-4fca-4318-af68-cba2f86e3932" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036353 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="2912b3cb-3ec7-4159-b63c-edc7391fa06a" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036361 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="77fc5a60-3070-4904-a9c1-c8a4b6d9d123" containerName="payload-pre-processing" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036370 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcb80707-e293-4a11-ab07-a5700d3abc57" containerName="istio-proxy" Jul 24 23:07:02.036504 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.036379 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="723b83ef-3f72-4ae0-a8dc-6027cc515325" containerName="payload-processing" Jul 24 23:07:02.039444 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.039425 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.044265 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.043714 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rsljv\"/\"openshift-service-ca.crt\"" Jul 24 23:07:02.044407 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.043746 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rsljv\"/\"kube-root-ca.crt\"" Jul 24 23:07:02.044407 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.043773 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rsljv\"/\"default-dockercfg-66cxv\"" Jul 24 23:07:02.046452 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.046136 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g"] Jul 24 23:07:02.152654 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.152606 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-sys\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.152654 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.152661 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-lib-modules\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.152929 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.152691 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6fd2\" (UniqueName: \"kubernetes.io/projected/2b1929e9-e914-467b-98c4-db77966ee381-kube-api-access-s6fd2\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.152929 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.152728 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-podres\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.152929 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.152760 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-proc\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253447 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253411 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-lib-modules\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253458 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6fd2\" (UniqueName: \"kubernetes.io/projected/2b1929e9-e914-467b-98c4-db77966ee381-kube-api-access-s6fd2\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253481 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-podres\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253517 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-proc\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253542 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-sys\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253587 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-lib-modules\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253640 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253613 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-proc\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253904 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253647 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-sys\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.253904 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.253652 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2b1929e9-e914-467b-98c4-db77966ee381-podres\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.262332 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.262312 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6fd2\" (UniqueName: \"kubernetes.io/projected/2b1929e9-e914-467b-98c4-db77966ee381-kube-api-access-s6fd2\") pod \"perf-node-gather-daemonset-c2g7g\" (UID: \"2b1929e9-e914-467b-98c4-db77966ee381\") " pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.350271 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.350187 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:02.499611 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:02.499551 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g"] Jul 24 23:07:02.501853 ip-10-0-143-53 kubenswrapper[2554]: W0724 23:07:02.501827 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2b1929e9_e914_467b_98c4_db77966ee381.slice/crio-794ad902889a67e7e8dd0b20e852373232d4e8c52430ca25a7c477aa56a0d376 WatchSource:0}: Error finding container 794ad902889a67e7e8dd0b20e852373232d4e8c52430ca25a7c477aa56a0d376: Status 404 returned error can't find the container with id 794ad902889a67e7e8dd0b20e852373232d4e8c52430ca25a7c477aa56a0d376 Jul 24 23:07:03.271072 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:03.271037 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" event={"ID":"2b1929e9-e914-467b-98c4-db77966ee381","Type":"ContainerStarted","Data":"86b2c3418acf2254d8c2f3ec64bebd646bdbedcbda443fe21345c7c5808196ad"} Jul 24 23:07:03.271521 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:03.271094 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" event={"ID":"2b1929e9-e914-467b-98c4-db77966ee381","Type":"ContainerStarted","Data":"794ad902889a67e7e8dd0b20e852373232d4e8c52430ca25a7c477aa56a0d376"} Jul 24 23:07:03.271521 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:03.271145 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:03.286421 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:03.286371 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" podStartSLOduration=1.2863568939999999 podStartE2EDuration="1.286356894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 23:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 23:07:03.285688194 +0000 UTC m=+3120.672506261" watchObservedRunningTime="2026-07-24 23:07:03.286356894 +0000 UTC m=+3120.673174939" Jul 24 23:07:03.581936 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:03.581843 2554 scope.go:117] "RemoveContainer" containerID="d7c35c4e98005c73c0f11f841c14a2163fc9f5487ae2fd56328d89110006e348" Jul 24 23:07:04.735507 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:04.735477 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7ln87_3bd4a67d-4043-43f8-bdfd-cdb12975a5e8/dns/0.log" Jul 24 23:07:04.751698 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:04.751673 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7ln87_3bd4a67d-4043-43f8-bdfd-cdb12975a5e8/kube-rbac-proxy/0.log" Jul 24 23:07:04.848645 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:04.848619 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6848f_a537a1d5-cc48-4106-85bb-5799059981b2/dns-node-resolver/0.log" Jul 24 23:07:05.419402 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:05.419377 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-95k6d_8eb6f6a5-436c-4a1b-9333-ca67a6e50c53/node-ca/0.log" Jul 24 23:07:06.332442 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:06.332413 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq_9549c3dc-c2ef-4c2e-a716-11d3338e3234/istio-proxy/0.log" Jul 24 23:07:06.472931 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:06.472903 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-phlct_68dbf04d-ba00-4c10-bfd8-2e57db942711/discovery/0.log" Jul 24 23:07:06.626083 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:06.625996 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cbc94_19a1f037-a1ee-4488-968c-ff384c53287d/payload-processing/0.log" Jul 24 23:07:06.646572 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:06.646543 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-858d756bd9-xqm8h_43c537e5-5787-4685-8933-340514153061/router/0.log" Jul 24 23:07:07.456211 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:07.456186 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jr455_b1604448-3b43-4289-a7da-abfe3cb9b4c1/serve-healthcheck-canary/0.log" Jul 24 23:07:07.908571 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:07.908535 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-mm46z_36fd0bd6-9a6b-4676-844c-0802bce022a5/insights-operator/0.log" Jul 24 23:07:07.909954 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:07.909938 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-mm46z_36fd0bd6-9a6b-4676-844c-0802bce022a5/insights-operator/1.log" Jul 24 23:07:08.034990 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:08.034961 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wl6px_90cc2e83-c8f6-4f31-8bdc-389922945c35/kube-rbac-proxy/0.log" Jul 24 23:07:08.050030 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:08.050005 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wl6px_90cc2e83-c8f6-4f31-8bdc-389922945c35/exporter/0.log" Jul 24 23:07:08.066678 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:08.066654 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wl6px_90cc2e83-c8f6-4f31-8bdc-389922945c35/extractor/0.log" Jul 24 23:07:09.285708 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:09.285680 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rsljv/perf-node-gather-daemonset-c2g7g" Jul 24 23:07:10.392366 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:10.392333 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-656df4d8c4-g286b_e3d0e7fa-1e6b-407a-b972-ceac7be0480b/manager/0.log" Jul 24 23:07:10.698963 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:10.698939 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-854fb8676-ptctb_48e11fca-1e70-41de-b8c3-5ebe02ea63a5/manager/0.log" Jul 24 23:07:10.711603 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:10.711584 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-854fb8676-ptctb_48e11fca-1e70-41de-b8c3-5ebe02ea63a5/copy-manifests/0.log" Jul 24 23:07:12.139627 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:12.139605 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b958b6f7b-qgb8r_614a5ea8-1e97-40ac-b72a-b7748929af94/manager/0.log" Jul 24 23:07:12.162852 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:12.162827 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-85swl_34a5c9c8-0c14-4d6a-a8a4-a5cc3a333c38/openshift-lws-operator/0.log" Jul 24 23:07:12.776903 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:12.776872 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq_9549c3dc-c2ef-4c2e-a716-11d3338e3234/istio-proxy/0.log" Jul 24 23:07:12.922559 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:12.922526 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-phlct_68dbf04d-ba00-4c10-bfd8-2e57db942711/discovery/0.log" Jul 24 23:07:13.053358 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:13.053291 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cbc94_19a1f037-a1ee-4488-968c-ff384c53287d/payload-processing/0.log" Jul 24 23:07:13.067657 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:13.067634 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-858d756bd9-xqm8h_43c537e5-5787-4685-8933-340514153061/router/0.log" Jul 24 23:07:14.442067 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:14.442037 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-gqmsq_9549c3dc-c2ef-4c2e-a716-11d3338e3234/istio-proxy/0.log" Jul 24 23:07:14.585167 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:14.585130 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-phlct_68dbf04d-ba00-4c10-bfd8-2e57db942711/discovery/0.log" Jul 24 23:07:14.716974 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:14.716894 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-cbc94_19a1f037-a1ee-4488-968c-ff384c53287d/payload-processing/0.log" Jul 24 23:07:14.730202 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:14.730175 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-858d756bd9-xqm8h_43c537e5-5787-4685-8933-340514153061/router/0.log" Jul 24 23:07:17.087801 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:17.087772 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tzzf5_f4af5df1-f958-47f0-9304-3cb39f18cf42/migrator/0.log" Jul 24 23:07:17.105342 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:17.105314 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tzzf5_f4af5df1-f958-47f0-9304-3cb39f18cf42/graceful-termination/0.log" Jul 24 23:07:18.687969 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.687939 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/kube-multus-additional-cni-plugins/0.log" Jul 24 23:07:18.704317 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.704294 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/egress-router-binary-copy/0.log" Jul 24 23:07:18.721016 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.720997 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/cni-plugins/0.log" Jul 24 23:07:18.739662 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.739636 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/bond-cni-plugin/0.log" Jul 24 23:07:18.759321 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.759302 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/routeoverride-cni/0.log" Jul 24 23:07:18.775849 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.775831 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/whereabouts-cni-bincopy/0.log" Jul 24 23:07:18.791568 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.791549 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4tnc_08821459-00c8-4cca-991b-e092689381e3/whereabouts-cni/0.log" Jul 24 23:07:18.824016 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.823993 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cxdjj_65765a0f-dd2d-4bca-a6cf-7bd77ead7425/kube-multus/0.log" Jul 24 23:07:18.923110 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.923082 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kpxxn_3429398f-e14d-41cc-aa66-70b22fa7e963/network-metrics-daemon/0.log" Jul 24 23:07:18.936196 ip-10-0-143-53 kubenswrapper[2554]: I0724 23:07:18.936168 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kpxxn_3429398f-e14d-41cc-aa66-70b22fa7e963/kube-rbac-proxy/0.log"