Jul 24 13:43:07.377291 ip-10-0-143-204 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 24 13:43:07.377306 ip-10-0-143-204 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 24 13:43:07.377315 ip-10-0-143-204 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 24 13:43:07.377906 ip-10-0-143-204 systemd[1]: Failed to start Kubernetes Kubelet. Jul 24 13:43:17.404482 ip-10-0-143-204 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 24 13:43:17.404502 ip-10-0-143-204 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8504d498cbe94677b9466fca0150f287 -- Jul 24 13:45:13.085319 ip-10-0-143-204 systemd[1]: Starting Kubernetes Kubelet... Jul 24 13:45:13.470855 ip-10-0-143-204 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:45:13.470855 ip-10-0-143-204 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 13:45:13.470855 ip-10-0-143-204 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:45:13.470855 ip-10-0-143-204 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 13:45:13.470855 ip-10-0-143-204 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:45:13.472530 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.472439 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 13:45:13.476851 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476832 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:45:13.476851 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476851 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476855 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476858 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476861 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476864 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476867 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476873 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476876 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476880 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476884 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476888 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476891 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476894 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476897 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476899 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476902 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476904 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476907 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476911 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:45:13.476925 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476914 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476916 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476919 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476921 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476924 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476927 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476929 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476933 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476936 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476939 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476942 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476944 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476947 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476950 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476952 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476956 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476959 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476962 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476965 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:45:13.477397 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476968 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476971 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476974 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476976 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476991 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476994 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476997 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.476999 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477001 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477004 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477007 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477009 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477013 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477016 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477018 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477021 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477024 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477027 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477029 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477032 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:45:13.477846 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477035 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477037 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477040 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477043 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477046 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477048 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477051 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477053 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477055 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477058 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477060 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477064 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477067 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477070 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477072 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477075 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477077 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477082 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477085 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:45:13.478361 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477089 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477091 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477094 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477097 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477100 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477102 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477105 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477107 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477111 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477113 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477116 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477119 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477121 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477124 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477126 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477129 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477131 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477134 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477137 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477140 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:45:13.478829 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477142 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477844 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477850 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477855 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477858 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477861 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477865 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477867 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477870 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477873 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477875 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477878 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477880 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477883 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477885 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477888 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477890 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477892 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477895 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:45:13.479400 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477898 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477901 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477904 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477907 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477909 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477912 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477915 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477917 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477920 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477922 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477925 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477928 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477930 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477933 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477937 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477941 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477944 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477947 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477950 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477952 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:45:13.479877 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477956 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477959 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477961 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477964 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477966 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477969 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477972 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477974 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477977 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.477996 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478000 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478004 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478008 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478012 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478015 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478018 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478021 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478023 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478026 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478029 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:45:13.480377 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478031 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478034 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478037 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478039 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478041 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478044 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478047 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478049 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478052 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478055 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478057 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478060 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478062 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478067 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478069 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478072 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478074 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478077 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478079 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:45:13.480891 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478082 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478084 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478087 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478089 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478093 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478096 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478099 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478102 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478104 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478107 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478111 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478114 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478118 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478121 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478124 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478127 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478130 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478132 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478135 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478138 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:45:13.481417 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478140 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478143 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478607 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478612 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478616 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478619 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478622 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478627 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478630 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478633 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478635 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478638 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478640 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478643 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478645 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478649 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478653 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478655 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478659 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:45:13.481896 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478661 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478664 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478667 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478670 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478673 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478676 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478678 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478681 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478684 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478686 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478689 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478691 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478694 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478696 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478699 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478701 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478703 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478706 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478708 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478711 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:45:13.482385 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478714 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478717 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478720 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478722 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478725 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478727 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478730 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478732 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478734 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478737 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478740 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478743 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478745 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478748 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478751 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478753 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478756 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478758 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478761 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478763 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:45:13.482867 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478765 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478768 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478772 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478775 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478778 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478781 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478784 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478787 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478789 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478792 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478794 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478797 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478799 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478802 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478805 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478807 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478809 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478811 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478814 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:45:13.483405 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478816 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478819 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478821 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478826 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478829 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478832 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478835 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478837 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478840 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478843 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478845 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478848 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478850 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478853 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478855 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478858 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478860 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478863 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478865 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478867 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:45:13.483871 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478870 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478872 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.478875 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.478953 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.478964 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.478974 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.478997 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479006 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479010 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479015 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479019 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479022 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479025 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479029 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479032 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479037 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479042 2555 flags.go:64] FLAG: --cgroup-root="" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479045 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479048 2555 flags.go:64] FLAG: --client-ca-file="" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479050 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479053 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479058 2555 flags.go:64] FLAG: --cluster-domain="" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479061 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 13:45:13.484450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479065 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479068 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479071 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479075 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479078 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479081 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479084 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479087 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479090 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479093 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479096 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479099 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479103 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479106 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479109 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479112 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479115 2555 flags.go:64] FLAG: --enable-server="true" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479119 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479123 2555 flags.go:64] FLAG: --event-burst="100" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479126 2555 flags.go:64] FLAG: --event-qps="50" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479129 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479132 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479135 2555 flags.go:64] FLAG: --eviction-hard="" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479139 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 13:45:13.485022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479144 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479148 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479151 2555 flags.go:64] FLAG: --eviction-soft="" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479154 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479157 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479160 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479163 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479166 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479168 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479171 2555 flags.go:64] FLAG: --feature-gates="" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479175 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479178 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479181 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479184 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479187 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479190 2555 flags.go:64] FLAG: --help="false" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479193 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479196 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479199 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479202 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479205 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479209 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479212 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 13:45:13.485605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479215 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479217 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479220 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479224 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479227 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479230 2555 flags.go:64] FLAG: --kube-reserved="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479232 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479235 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479238 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479242 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479246 2555 flags.go:64] FLAG: --lock-file="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479249 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479252 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479255 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479260 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479263 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479266 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479269 2555 flags.go:64] FLAG: --logging-format="text" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479271 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479275 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479278 2555 flags.go:64] FLAG: --manifest-url="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479281 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479286 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479289 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479293 2555 flags.go:64] FLAG: --max-pods="110" Jul 24 13:45:13.486223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479296 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479299 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479302 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479305 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479308 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479311 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479314 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479322 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479325 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479328 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479331 2555 flags.go:64] FLAG: --pod-cidr="" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479334 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479347 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479350 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479353 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479356 2555 flags.go:64] FLAG: --port="10250" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479360 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479363 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03252ddd2d8d5da3d" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479368 2555 flags.go:64] FLAG: --qos-reserved="" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479371 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479374 2555 flags.go:64] FLAG: --register-node="true" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479377 2555 flags.go:64] FLAG: --register-with-taints="" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479384 2555 flags.go:64] FLAG: --registry-burst="10" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479387 2555 flags.go:64] FLAG: --registry-qps="5" Jul 24 13:45:13.486844 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479390 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479393 2555 flags.go:64] FLAG: --reserved-memory="" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479397 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479400 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479403 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479406 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479409 2555 flags.go:64] FLAG: --runonce="false" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479411 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479414 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479417 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479420 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479423 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479426 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479429 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479431 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479435 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479438 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479441 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479443 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479446 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479450 2555 flags.go:64] FLAG: --system-cgroups="" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479453 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479458 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479460 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479465 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 13:45:13.487440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479469 2555 flags.go:64] FLAG: --tls-min-version="" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479472 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479475 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479478 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479481 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479484 2555 flags.go:64] FLAG: --v="2" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479489 2555 flags.go:64] FLAG: --version="false" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479493 2555 flags.go:64] FLAG: --vmodule="" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479496 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479499 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479622 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479625 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479628 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479631 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479634 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479636 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479639 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479642 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479644 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479647 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479650 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:45:13.488099 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479652 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479655 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479657 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479660 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479664 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479667 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479670 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479674 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479678 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479681 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479685 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479688 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479691 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479694 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479697 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479700 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479702 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479705 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479707 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:45:13.488611 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479710 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479713 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479715 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479718 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479720 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479722 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479725 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479727 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479729 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479732 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479734 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479737 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479739 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479741 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479744 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479746 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479749 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479751 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479754 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479756 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:45:13.489109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479759 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479762 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479764 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479768 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479770 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479773 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479775 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479778 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479780 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479783 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479786 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479788 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479791 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479793 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479796 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479798 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479801 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479803 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479805 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479808 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:45:13.489598 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479810 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479813 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479815 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479818 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479820 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479822 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479825 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479827 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479829 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479832 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479834 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479837 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479839 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479841 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479844 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479848 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479851 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479853 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479856 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479858 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:45:13.490109 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479861 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479865 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479868 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479871 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479873 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479876 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479878 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479881 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.479884 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.479893 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.487735 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.487753 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.487778 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.487787 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 13:45:13.490605 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487861 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487868 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487872 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487875 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487878 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487882 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487885 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487888 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487891 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487893 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487896 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487899 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487910 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487912 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487915 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487918 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487920 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487923 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487927 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487931 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:45:13.490960 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487935 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487939 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487942 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487945 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487947 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487950 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487957 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487960 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487962 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487966 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487968 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487971 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487974 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487976 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.487995 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488000 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488004 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488006 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488009 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488012 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488015 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:45:13.491478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488017 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488021 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488023 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488025 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488035 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488037 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488040 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488042 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488045 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488050 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488054 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488057 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488060 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488063 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488068 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488072 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488077 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488081 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488084 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:45:13.492006 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488087 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488090 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488093 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488097 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488100 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488104 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488107 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488110 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488113 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488122 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488124 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488127 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488129 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488132 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488134 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488136 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488139 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488141 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488157 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:45:13.492478 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488161 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488164 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488167 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488169 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488172 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488174 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488177 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488179 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488181 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488184 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488186 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488189 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488191 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488194 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488197 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488199 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488201 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488204 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488206 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:45:13.492935 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488210 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.488216 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488398 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488405 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488408 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488412 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488416 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488419 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488422 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488426 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488428 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488431 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488433 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488442 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:45:13.493564 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488445 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488448 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488450 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488453 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488456 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488459 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488464 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488468 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488472 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488476 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488478 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488481 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488483 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488486 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488488 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488491 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488493 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488496 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488498 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488501 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:45:13.493917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488503 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488506 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488508 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488510 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488513 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488515 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488518 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488520 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488523 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488525 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488527 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488530 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488542 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488547 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488553 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488556 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488558 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488561 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488563 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488565 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:45:13.494428 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488568 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488570 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488573 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488575 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488578 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488580 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488583 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488586 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488588 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488591 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488593 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488595 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488598 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488600 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488603 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488605 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488608 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488610 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488613 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488617 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:45:13.494916 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488621 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488626 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488629 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488632 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488634 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488643 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488646 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488649 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488651 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488654 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488656 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488658 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488661 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488663 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488666 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488668 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488671 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488673 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488676 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488679 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:45:13.495440 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488681 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488684 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488686 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488689 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488691 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488695 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:13.488699 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.488707 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.488855 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.491088 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.492408 2555 server.go:1019] "Starting client certificate rotation" Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.492523 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.493348 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:45:13.495922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.495019 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:45:13.499283 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.499256 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:45:13.513231 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.513202 2555 log.go:25] "Validated CRI v1 runtime API" Jul 24 13:45:13.515383 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.515354 2555 log.go:25] "Validated CRI v1 image API" Jul 24 13:45:13.516943 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.516928 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 13:45:13.522411 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.522383 2555 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a8471286-774a-4e4f-96c6-317f6925cfa0:/dev/nvme0n1p4 e4df3e1d-c01a-4ca9-9a16-cd8028796e5a:/dev/nvme0n1p3] Jul 24 13:45:13.522476 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.522410 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 13:45:13.523209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.523189 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:45:13.526997 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.526874 2555 manager.go:216] Machine: {Timestamp:2026-07-24 13:45:13.526348209 +0000 UTC m=+0.321690988 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095459 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cd84d1ef389116fda2b1ce89b5f64 SystemUUID:ec2cd84d-1ef3-8911-6fda-2b1ce89b5f64 BootID:8504d498-cbe9-4677-b946-6fca0150f287 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:85:f0:94:ec:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:85:f0:94:ec:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:d8:4d:08:e7:88 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 13:45:13.526997 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.526993 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 13:45:13.527117 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.527082 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 13:45:13.530199 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.530167 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 13:45:13.530390 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.530202 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 13:45:13.530970 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.530960 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 13:45:13.531042 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.530973 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 13:45:13.531042 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.530994 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:45:13.531668 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.531657 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:45:13.532575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.532566 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 13:45:13.534581 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534566 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:45:13.534711 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534701 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 13:45:13.534752 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534740 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 24 13:45:13.534781 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534753 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 13:45:13.534781 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534769 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 13:45:13.534781 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534780 2555 kubelet.go:398] "Adding apiserver pod source" Jul 24 13:45:13.534871 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.534790 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 13:45:13.535240 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.535227 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:45:13.535302 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.535246 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:45:13.538117 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.538092 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 13:45:13.539941 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.539923 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 13:45:13.540059 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.539963 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 13:45:13.541601 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541576 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 13:45:13.541601 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541602 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541609 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541616 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541625 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541632 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541642 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541652 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541662 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541667 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541676 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 13:45:13.541719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.541681 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 13:45:13.542553 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.542543 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 13:45:13.542553 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.542552 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 13:45:13.546701 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.546666 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xk4d8" Jul 24 13:45:13.547389 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.547260 2555 server.go:1268] "Started kubelet" Jul 24 13:45:13.547389 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.547304 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 13:45:13.547542 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.547381 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 13:45:13.547542 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.547450 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 13:45:13.548611 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.548560 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 13:45:13.548624 ip-10-0-143-204 systemd[1]: Started Kubernetes Kubelet. Jul 24 13:45:13.549589 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.549530 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 13:45:13.550444 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.549299 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 13:45:13.550574 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.549279 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 13:45:13.550711 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.550579 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xk4d8" Jul 24 13:45:13.551043 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.551027 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 24 13:45:13.554107 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.554088 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 13:45:13.554700 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.553794 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-204.ec2.internal.18c53dbdfe4f67d6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-204.ec2.internal,UID:ip-10-0-143-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-204.ec2.internal,},FirstTimestamp:2026-07-24 13:45:13.546999766 +0000 UTC m=+0.342342474,LastTimestamp:2026-07-24 13:45:13.546999766 +0000 UTC m=+0.342342474,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-204.ec2.internal,}" Jul 24 13:45:13.558070 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558045 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 13:45:13.558070 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558056 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 13:45:13.558700 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.558167 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:13.558700 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558607 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 13:45:13.558700 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558624 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 13:45:13.558700 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558672 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 13:45:13.558879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558742 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 24 13:45:13.558879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.558750 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 24 13:45:13.559033 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559016 2555 factory.go:55] Registering systemd factory Jul 24 13:45:13.559116 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559038 2555 factory.go:223] Registration of the systemd container factory successfully Jul 24 13:45:13.559276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559255 2555 factory.go:153] Registering CRI-O factory Jul 24 13:45:13.559276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559276 2555 factory.go:223] Registration of the crio container factory successfully Jul 24 13:45:13.559393 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559322 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 13:45:13.559393 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559347 2555 factory.go:103] Registering Raw factory Jul 24 13:45:13.559393 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559360 2555 manager.go:1196] Started watching for new ooms in manager Jul 24 13:45:13.559845 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.559832 2555 manager.go:349] Starting recovery of all containers Jul 24 13:45:13.567659 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.567524 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:45:13.567857 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.567809 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 13:45:13.570906 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.570889 2555 manager.go:354] Recovery completed Jul 24 13:45:13.571370 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.571350 2555 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-204.ec2.internal\" not found" node="ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.576932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.576918 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.578784 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.578766 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.578866 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.578793 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.578866 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.578804 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.579298 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.579286 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 13:45:13.579355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.579298 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 13:45:13.579355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.579318 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:45:13.579355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.579344 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.580090 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.580074 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.580130 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.580107 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.580130 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.580119 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.581022 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.581009 2555 policy_none.go:49] "None policy: Start" Jul 24 13:45:13.581060 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.581028 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 13:45:13.581060 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.581041 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 13:45:13.582320 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.582309 2555 policy_none.go:47] "Start" Jul 24 13:45:13.622836 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.622806 2555 manager.go:337] "Starting Device Plugin manager" Jul 24 13:45:13.622971 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.622852 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 13:45:13.622971 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.622866 2555 server.go:87] "Starting device plugin registration server" Jul 24 13:45:13.623196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.623182 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 13:45:13.623234 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.623196 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 13:45:13.623319 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.623297 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 13:45:13.623537 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.623414 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 13:45:13.623537 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.623427 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 13:45:13.624002 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.623966 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 13:45:13.624072 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.624041 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:13.724306 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.724206 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.726296 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.726279 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.726396 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.726309 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.726396 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.726321 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.726396 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.726352 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.733638 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.733621 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.733683 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.733646 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-204.ec2.internal\": node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:13.742589 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.742562 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 13:45:13.742711 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.742606 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 13:45:13.742711 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.742633 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 13:45:13.742711 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.742672 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 13:45:13.744350 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.744325 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:45:13.747189 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.747170 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:13.843290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.843236 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal"] Jul 24 13:45:13.843456 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.843334 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.844355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.844341 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.844418 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.844371 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.844418 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.844382 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.845637 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.845624 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.845755 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.845732 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.845802 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.845769 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.846354 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.846334 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.846467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.846335 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.846467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.846370 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.846467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.846381 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.846467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.846399 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.846467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.846418 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.847721 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.847703 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.847797 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.847731 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:13.847797 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.847754 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:45:13.848447 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.848417 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:45:13.848522 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.848461 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:45:13.848522 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.848473 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:45:13.859148 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.859125 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fde6aac7dcc7c6e378a0f2cd26c09763-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal\" (UID: \"fde6aac7dcc7c6e378a0f2cd26c09763\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.859254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.859156 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fde6aac7dcc7c6e378a0f2cd26c09763-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal\" (UID: \"fde6aac7dcc7c6e378a0f2cd26c09763\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.859254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.859178 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2b0af06f1bfda6fafb07193c56cc418-config\") pod \"kube-apiserver-proxy-ip-10-0-143-204.ec2.internal\" (UID: \"b2b0af06f1bfda6fafb07193c56cc418\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.874938 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.874908 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-204.ec2.internal\" not found" node="ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.879450 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.879430 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-204.ec2.internal\" not found" node="ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.947817 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:13.947782 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:13.960116 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.960087 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2b0af06f1bfda6fafb07193c56cc418-config\") pod \"kube-apiserver-proxy-ip-10-0-143-204.ec2.internal\" (UID: \"b2b0af06f1bfda6fafb07193c56cc418\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.960215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.960126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fde6aac7dcc7c6e378a0f2cd26c09763-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal\" (UID: \"fde6aac7dcc7c6e378a0f2cd26c09763\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.960215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.960152 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fde6aac7dcc7c6e378a0f2cd26c09763-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal\" (UID: \"fde6aac7dcc7c6e378a0f2cd26c09763\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.960215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.960179 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2b0af06f1bfda6fafb07193c56cc418-config\") pod \"kube-apiserver-proxy-ip-10-0-143-204.ec2.internal\" (UID: \"b2b0af06f1bfda6fafb07193c56cc418\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.960215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.960186 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fde6aac7dcc7c6e378a0f2cd26c09763-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal\" (UID: \"fde6aac7dcc7c6e378a0f2cd26c09763\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:13.960215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:13.960194 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fde6aac7dcc7c6e378a0f2cd26c09763-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal\" (UID: \"fde6aac7dcc7c6e378a0f2cd26c09763\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:14.048561 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.048494 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.149012 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.148956 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.178446 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.178424 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:14.182497 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.182384 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" Jul 24 13:45:14.249924 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.249867 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.350489 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.350392 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.451073 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.451023 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.491489 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.491457 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 13:45:14.492076 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.491618 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:45:14.492076 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.491635 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:45:14.551634 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.551595 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.556528 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.556496 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 13:40:13 +0000 UTC" deadline="2028-01-05 17:20:01.966265267 +0000 UTC" Jul 24 13:45:14.556582 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.556528 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12723h34m47.409740425s" Jul 24 13:45:14.652646 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.652622 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.753070 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:14.753027 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-204.ec2.internal\" not found" Jul 24 13:45:14.845803 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.845773 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:45:14.858908 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.858879 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" Jul 24 13:45:14.872329 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.872302 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:45:14.874067 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.874037 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" Jul 24 13:45:14.883984 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.883954 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:45:14.916893 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:14.916847 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b0af06f1bfda6fafb07193c56cc418.slice/crio-5c2f7aa319aeeab2209c8bc464b81238504494478bdda216a9df8c9730e2bef5 WatchSource:0}: Error finding container 5c2f7aa319aeeab2209c8bc464b81238504494478bdda216a9df8c9730e2bef5: Status 404 returned error can't find the container with id 5c2f7aa319aeeab2209c8bc464b81238504494478bdda216a9df8c9730e2bef5 Jul 24 13:45:14.922338 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.922321 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:45:14.940634 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:14.940602 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde6aac7dcc7c6e378a0f2cd26c09763.slice/crio-c57482882d8c2963232fcdb56b143147acf81b808d25b9cfb8e0076286e116c3 WatchSource:0}: Error finding container c57482882d8c2963232fcdb56b143147acf81b808d25b9cfb8e0076286e116c3: Status 404 returned error can't find the container with id c57482882d8c2963232fcdb56b143147acf81b808d25b9cfb8e0076286e116c3 Jul 24 13:45:14.978687 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:14.978655 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:45:15.536147 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.536109 2555 apiserver.go:52] "Watching apiserver" Jul 24 13:45:15.548943 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.548907 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 13:45:15.549474 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.549435 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wsr7q","kube-system/konnectivity-agent-9228l","kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal","openshift-dns/node-resolver-cvnzc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal","openshift-multus/multus-xpkrh","openshift-multus/network-metrics-daemon-ql85p","openshift-network-operator/iptables-alerter-kjr4v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p","openshift-cluster-node-tuning-operator/tuned-pjdwk","openshift-image-registry/node-ca-n9vmm","openshift-multus/multus-additional-cni-plugins-kb9dl","openshift-network-diagnostics/network-check-target-p9p9n","openshift-ovn-kubernetes/ovnkube-node-b8jcw"] Jul 24 13:45:15.551313 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.551284 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:15.551427 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.551399 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:15.552826 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.552802 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.554402 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.555568 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.555748 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.555961 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8ghhj\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.556469 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.556496 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.557023 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.557263 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.557399 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 13:45:15.557604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.557463 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q6h2c\"" Jul 24 13:45:15.558209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.557672 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.558209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.558087 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.558781 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.558760 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ddj79\"" Jul 24 13:45:15.558997 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.558974 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.559263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.559238 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 13:45:15.559338 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.559292 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.559927 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.559905 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.560206 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.560184 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.560480 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.560459 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxn78\"" Jul 24 13:45:15.560561 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.560457 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.560712 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.560678 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 13:45:15.560788 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.560753 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 13:45:15.560852 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.560793 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 13:45:15.561759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.561740 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 13:45:15.561925 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.561911 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p49bp\"" Jul 24 13:45:15.562225 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.562204 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:15.562307 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.562278 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:15.562685 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.562664 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 13:45:15.562685 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.562682 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jmhln\"" Jul 24 13:45:15.562917 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.562897 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 13:45:15.563939 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.563918 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.565187 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.565162 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.566562 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.566534 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8j6h7\"" Jul 24 13:45:15.566671 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.566564 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.566814 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.566798 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.566912 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.566863 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:15.567112 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.567080 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 13:45:15.567344 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.567323 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.567652 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.567633 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9ppln\"" Jul 24 13:45:15.568525 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.567918 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.568525 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568071 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.568525 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568205 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.568729 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568679 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhmwq\" (UniqueName: \"kubernetes.io/projected/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-kube-api-access-lhmwq\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:15.568729 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-os-release\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.568848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568741 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-hostroot\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.568848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568768 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-os-release\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.568848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421b243e-5770-4c8b-985b-6f58e9e8899e-cni-binary-copy\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.568848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568830 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.568848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d4d8d2-be23-425c-b318-811a3cd71ab8-tmp-dir\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568880 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzcbz\" (UniqueName: \"kubernetes.io/projected/01d4d8d2-be23-425c-b318-811a3cd71ab8-kube-api-access-dzcbz\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568909 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-netns\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568949 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-cni-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568974 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-cnibin\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.568998 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc7xw\" (UniqueName: \"kubernetes.io/projected/7a02ec45-abb1-4162-b8e1-b9fd31567f85-kube-api-access-hc7xw\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569032 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-conf-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-daemon-config\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569101 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqmdf\" (UniqueName: \"kubernetes.io/projected/421b243e-5770-4c8b-985b-6f58e9e8899e-kube-api-access-zqmdf\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569145 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f085458a-3b25-43ca-85a4-d79e0ae4a2f8-agent-certs\") pod \"konnectivity-agent-9228l\" (UID: \"f085458a-3b25-43ca-85a4-d79e0ae4a2f8\") " pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.569190 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569170 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a02ec45-abb1-4162-b8e1-b9fd31567f85-host-slash\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569200 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01d4d8d2-be23-425c-b318-811a3cd71ab8-hosts-file\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569248 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-system-cni-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569285 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-socket-dir-parent\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569323 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-cni-multus\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569348 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-kubelet\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569435 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct8tl\" (UniqueName: \"kubernetes.io/projected/be143ea6-d2ee-4709-8fd8-de79390abd1f-kube-api-access-ct8tl\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569463 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-k8s-cni-cncf-io\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569505 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a02ec45-abb1-4162-b8e1-b9fd31567f85-iptables-alerter-script\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569544 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-system-cni-dir\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569580 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be143ea6-d2ee-4709-8fd8-de79390abd1f-host\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569610 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-cni-bin\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-multus-certs\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.569724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-etc-kubernetes\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569756 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569780 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-cnibin\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569821 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569849 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bnjc\" (UniqueName: \"kubernetes.io/projected/f7a62685-2897-4199-bc3f-8f4471dbebd6-kube-api-access-8bnjc\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569882 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be143ea6-d2ee-4709-8fd8-de79390abd1f-serviceca\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.570397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.569905 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f085458a-3b25-43ca-85a4-d79e0ae4a2f8-konnectivity-ca\") pod \"konnectivity-agent-9228l\" (UID: \"f085458a-3b25-43ca-85a4-d79e0ae4a2f8\") " pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.570864 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.572322 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.572656 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.572786 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.573423 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.573632 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.573669 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mhlxc\"" Jul 24 13:45:15.575127 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.574120 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 13:45:15.596749 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.596716 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sl6jz" Jul 24 13:45:15.604635 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.604120 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sl6jz" Jul 24 13:45:15.660427 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.660362 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671060 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysctl-d\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671122 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f085458a-3b25-43ca-85a4-d79e0ae4a2f8-agent-certs\") pod \"konnectivity-agent-9228l\" (UID: \"f085458a-3b25-43ca-85a4-d79e0ae4a2f8\") " pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a02ec45-abb1-4162-b8e1-b9fd31567f85-host-slash\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671180 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01d4d8d2-be23-425c-b318-811a3cd71ab8-hosts-file\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-socket-dir-parent\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671287 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-cni-multus\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-kubelet\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct8tl\" (UniqueName: \"kubernetes.io/projected/be143ea6-d2ee-4709-8fd8-de79390abd1f-kube-api-access-ct8tl\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671385 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671411 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-442dg\" (UniqueName: \"kubernetes.io/projected/302ffe90-feec-4946-b2f8-885f77e67d18-kube-api-access-442dg\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671441 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671468 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-etc-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671492 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-log-socket\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671520 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a02ec45-abb1-4162-b8e1-b9fd31567f85-iptables-alerter-script\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671548 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-system-cni-dir\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.671981 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671574 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be143ea6-d2ee-4709-8fd8-de79390abd1f-host\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671598 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-cni-bin\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-etc-kubernetes\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671624 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671686 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/01d4d8d2-be23-425c-b318-811a3cd71ab8-hosts-file\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671727 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be143ea6-d2ee-4709-8fd8-de79390abd1f-serviceca\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671750 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-sys-fs\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671776 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-kubernetes\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysctl-conf\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-systemd\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671851 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-kubelet\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671877 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovnkube-config\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671907 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f085458a-3b25-43ca-85a4-d79e0ae4a2f8-konnectivity-ca\") pod \"konnectivity-agent-9228l\" (UID: \"f085458a-3b25-43ca-85a4-d79e0ae4a2f8\") " pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671908 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-socket-dir-parent\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671949 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7a02ec45-abb1-4162-b8e1-b9fd31567f85-host-slash\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671975 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-cni-multus\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.671953 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhmwq\" (UniqueName: \"kubernetes.io/projected/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-kube-api-access-lhmwq\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672027 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-kubelet\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.673036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-hostroot\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672111 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-hostroot\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672124 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/be143ea6-d2ee-4709-8fd8-de79390abd1f-host\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672167 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-socket-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672292 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672350 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-system-cni-dir\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672382 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-var-lib-cni-bin\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672569 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/be143ea6-d2ee-4709-8fd8-de79390abd1f-serviceca\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672627 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-run\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672651 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-var-lib-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672723 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovnkube-script-lib\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672746 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-node-log\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovn-node-metrics-cert\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672802 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d4d8d2-be23-425c-b318-811a3cd71ab8-tmp-dir\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672825 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzcbz\" (UniqueName: \"kubernetes.io/projected/01d4d8d2-be23-425c-b318-811a3cd71ab8-kube-api-access-dzcbz\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672848 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-netns\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672912 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f085458a-3b25-43ca-85a4-d79e0ae4a2f8-konnectivity-ca\") pod \"konnectivity-agent-9228l\" (UID: \"f085458a-3b25-43ca-85a4-d79e0ae4a2f8\") " pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672914 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7a02ec45-abb1-4162-b8e1-b9fd31567f85-iptables-alerter-script\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672917 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-registration-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672951 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-netns\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672960 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-device-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672987 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-etc-kubernetes\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.672992 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-tuned\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673129 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-run-netns\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673159 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-cnibin\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc7xw\" (UniqueName: \"kubernetes.io/projected/7a02ec45-abb1-4162-b8e1-b9fd31567f85-kube-api-access-hc7xw\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673232 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-conf-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673244 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-cnibin\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-daemon-config\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673315 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqmdf\" (UniqueName: \"kubernetes.io/projected/421b243e-5770-4c8b-985b-6f58e9e8899e-kube-api-access-zqmdf\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673325 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-conf-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673370 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.674715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673403 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-modprobe-d\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673467 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-kubelet-config\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673501 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-system-cni-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673526 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-dbus\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673553 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjgkk\" (UniqueName: \"kubernetes.io/projected/f5a273ff-bb8b-4756-9315-0db5130d47bc-kube-api-access-cjgkk\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673582 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-k8s-cni-cncf-io\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673284 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d4d8d2-be23-425c-b318-811a3cd71ab8-tmp-dir\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673655 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673725 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-multus-certs\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673810 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-sys\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673828 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-system-cni-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673836 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-daemon-config\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673896 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-k8s-cni-cncf-io\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.673841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-var-lib-kubelet\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674026 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-systemd-units\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674077 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-systemd\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674113 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-host-run-multus-certs\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674157 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-env-overrides\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674192 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.674279 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674321 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-cnibin\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674351 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674413 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674447 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bnjc\" (UniqueName: \"kubernetes.io/projected/f7a62685-2897-4199-bc3f-8f4471dbebd6-kube-api-access-8bnjc\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.674465 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:16.174442442 +0000 UTC m=+2.969785161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674635 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysconfig\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674666 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-lib-modules\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674696 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-os-release\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-slash\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674791 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-os-release\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421b243e-5770-4c8b-985b-6f58e9e8899e-cni-binary-copy\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-host\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.676457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674928 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5694c059-720d-48c0-808a-c2ce5c2c3685-tmp\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674959 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.674988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx67l\" (UniqueName: \"kubernetes.io/projected/5694c059-720d-48c0-808a-c2ce5c2c3685-kube-api-access-lx67l\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675018 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-cni-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-ovn\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675087 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-cni-bin\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675113 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-cni-netd\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675402 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675526 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-os-release\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675636 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421b243e-5770-4c8b-985b-6f58e9e8899e-cni-binary-copy\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675660 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675716 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-os-release\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675722 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421b243e-5770-4c8b-985b-6f58e9e8899e-multus-cni-dir\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675755 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7a62685-2897-4199-bc3f-8f4471dbebd6-cnibin\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.677303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.675956 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f7a62685-2897-4199-bc3f-8f4471dbebd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.680219 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.680192 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f085458a-3b25-43ca-85a4-d79e0ae4a2f8-agent-certs\") pod \"konnectivity-agent-9228l\" (UID: \"f085458a-3b25-43ca-85a4-d79e0ae4a2f8\") " pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.683884 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.683850 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzcbz\" (UniqueName: \"kubernetes.io/projected/01d4d8d2-be23-425c-b318-811a3cd71ab8-kube-api-access-dzcbz\") pod \"node-resolver-cvnzc\" (UID: \"01d4d8d2-be23-425c-b318-811a3cd71ab8\") " pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.684208 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.684187 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct8tl\" (UniqueName: \"kubernetes.io/projected/be143ea6-d2ee-4709-8fd8-de79390abd1f-kube-api-access-ct8tl\") pod \"node-ca-n9vmm\" (UID: \"be143ea6-d2ee-4709-8fd8-de79390abd1f\") " pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.684511 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.684485 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqmdf\" (UniqueName: \"kubernetes.io/projected/421b243e-5770-4c8b-985b-6f58e9e8899e-kube-api-access-zqmdf\") pod \"multus-xpkrh\" (UID: \"421b243e-5770-4c8b-985b-6f58e9e8899e\") " pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.686992 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.686564 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc7xw\" (UniqueName: \"kubernetes.io/projected/7a02ec45-abb1-4162-b8e1-b9fd31567f85-kube-api-access-hc7xw\") pod \"iptables-alerter-kjr4v\" (UID: \"7a02ec45-abb1-4162-b8e1-b9fd31567f85\") " pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.689516 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.689469 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhmwq\" (UniqueName: \"kubernetes.io/projected/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-kube-api-access-lhmwq\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:15.690281 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.690237 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bnjc\" (UniqueName: \"kubernetes.io/projected/f7a62685-2897-4199-bc3f-8f4471dbebd6-kube-api-access-8bnjc\") pod \"multus-additional-cni-plugins-kb9dl\" (UID: \"f7a62685-2897-4199-bc3f-8f4471dbebd6\") " pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.748151 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.748080 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" event={"ID":"fde6aac7dcc7c6e378a0f2cd26c09763","Type":"ContainerStarted","Data":"c57482882d8c2963232fcdb56b143147acf81b808d25b9cfb8e0076286e116c3"} Jul 24 13:45:15.749813 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.749780 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" event={"ID":"b2b0af06f1bfda6fafb07193c56cc418","Type":"ContainerStarted","Data":"5c2f7aa319aeeab2209c8bc464b81238504494478bdda216a9df8c9730e2bef5"} Jul 24 13:45:15.775978 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.775939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5694c059-720d-48c0-808a-c2ce5c2c3685-tmp\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.775993 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx67l\" (UniqueName: \"kubernetes.io/projected/5694c059-720d-48c0-808a-c2ce5c2c3685-kube-api-access-lx67l\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776040 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-ovn\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-cni-bin\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776090 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-cni-netd\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776130 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysctl-d\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776142 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-cni-netd\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.776196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776190 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-ovn\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-442dg\" (UniqueName: \"kubernetes.io/projected/302ffe90-feec-4946-b2f8-885f77e67d18-kube-api-access-442dg\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776233 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-cni-bin\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776269 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-etc-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776293 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-log-socket\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-sys-fs\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-kubernetes\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysctl-conf\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776391 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-systemd\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776417 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-kubelet\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776440 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovnkube-config\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776466 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-socket-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.776510 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776488 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-run\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776551 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-run\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776593 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-etc-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776696 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-log-socket\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776730 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysctl-d\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776749 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-sys-fs\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776810 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-kubernetes\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776829 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-etc-selinux\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776919 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-kubelet\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.776956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776940 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysctl-conf\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.776978 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-systemd\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777118 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-var-lib-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777153 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovnkube-script-lib\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-node-log\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovn-node-metrics-cert\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-registration-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-device-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777306 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-tuned\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777331 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777356 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-run-netns\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777391 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777422 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-modprobe-d\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777449 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-kubelet-config\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777476 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-dbus\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777502 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjgkk\" (UniqueName: \"kubernetes.io/projected/f5a273ff-bb8b-4756-9315-0db5130d47bc-kube-api-access-cjgkk\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777531 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777557 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovnkube-config\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777586 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-sys\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777610 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-var-lib-kubelet\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777635 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-systemd-units\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777664 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-systemd\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777689 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-env-overrides\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777744 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysconfig\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-lib-modules\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777797 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-slash\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.777848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777825 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-host\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777926 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-host\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777932 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.777988 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-sys\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778043 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-var-lib-kubelet\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778114 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-systemd-units\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-systemd\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.778218 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.778284 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:45:16.278261026 +0000 UTC m=+3.073603743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-run-netns\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778418 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-modprobe-d\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778569 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-kubelet-config\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.778678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778653 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-run-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778885 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-dbus\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778942 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-slash\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.778997 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-sysconfig\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779061 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5694c059-720d-48c0-808a-c2ce5c2c3685-lib-modules\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779129 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-var-lib-openvswitch\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779229 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5694c059-720d-48c0-808a-c2ce5c2c3685-tmp\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779251 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-socket-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779298 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-node-log\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779354 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-registration-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779396 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5a273ff-bb8b-4756-9315-0db5130d47bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.779460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779451 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/302ffe90-feec-4946-b2f8-885f77e67d18-device-dir\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.780004 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.779797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-env-overrides\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.780352 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.780323 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovnkube-script-lib\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.782487 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.782461 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5694c059-720d-48c0-808a-c2ce5c2c3685-etc-tuned\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.782815 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.782781 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5a273ff-bb8b-4756-9315-0db5130d47bc-ovn-node-metrics-cert\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.784547 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.784522 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:15.784547 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.784554 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:15.784720 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.784569 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:15.784720 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:15.784632 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:45:16.284612501 +0000 UTC m=+3.079955211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:15.788309 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.788149 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx67l\" (UniqueName: \"kubernetes.io/projected/5694c059-720d-48c0-808a-c2ce5c2c3685-kube-api-access-lx67l\") pod \"tuned-pjdwk\" (UID: \"5694c059-720d-48c0-808a-c2ce5c2c3685\") " pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.789250 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.788667 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-442dg\" (UniqueName: \"kubernetes.io/projected/302ffe90-feec-4946-b2f8-885f77e67d18-kube-api-access-442dg\") pod \"aws-ebs-csi-driver-node-fpq4p\" (UID: \"302ffe90-feec-4946-b2f8-885f77e67d18\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.791679 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.791651 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjgkk\" (UniqueName: \"kubernetes.io/projected/f5a273ff-bb8b-4756-9315-0db5130d47bc-kube-api-access-cjgkk\") pod \"ovnkube-node-b8jcw\" (UID: \"f5a273ff-bb8b-4756-9315-0db5130d47bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.866767 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.866729 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n9vmm" Jul 24 13:45:15.878698 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.878664 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kjr4v" Jul 24 13:45:15.879917 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.879816 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe143ea6_d2ee_4709_8fd8_de79390abd1f.slice/crio-853679da46a76311ba220c77286cead53d87e4bf8a37f69c4ef92a65d9c14ef3 WatchSource:0}: Error finding container 853679da46a76311ba220c77286cead53d87e4bf8a37f69c4ef92a65d9c14ef3: Status 404 returned error can't find the container with id 853679da46a76311ba220c77286cead53d87e4bf8a37f69c4ef92a65d9c14ef3 Jul 24 13:45:15.889344 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.889313 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cvnzc" Jul 24 13:45:15.889831 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.889806 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a02ec45_abb1_4162_b8e1_b9fd31567f85.slice/crio-6257948cc8d2cd334f7c895b19705dcd4d3520767db3a574a3c08eb3289b6dd8 WatchSource:0}: Error finding container 6257948cc8d2cd334f7c895b19705dcd4d3520767db3a574a3c08eb3289b6dd8: Status 404 returned error can't find the container with id 6257948cc8d2cd334f7c895b19705dcd4d3520767db3a574a3c08eb3289b6dd8 Jul 24 13:45:15.890140 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.890110 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:45:15.898268 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.898241 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" Jul 24 13:45:15.904650 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.904621 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d4d8d2_be23_425c_b318_811a3cd71ab8.slice/crio-078217d21d19b7a199890562f3630c484c7ef6760a35790082be7b0fecb26035 WatchSource:0}: Error finding container 078217d21d19b7a199890562f3630c484c7ef6760a35790082be7b0fecb26035: Status 404 returned error can't find the container with id 078217d21d19b7a199890562f3630c484c7ef6760a35790082be7b0fecb26035 Jul 24 13:45:15.908863 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.908836 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xpkrh" Jul 24 13:45:15.910423 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.910397 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a62685_2897_4199_bc3f_8f4471dbebd6.slice/crio-0c2c8a1667da33c80dd3eb5c54ff7fa3cd3f571433ee2273ea93c12f633a7c17 WatchSource:0}: Error finding container 0c2c8a1667da33c80dd3eb5c54ff7fa3cd3f571433ee2273ea93c12f633a7c17: Status 404 returned error can't find the container with id 0c2c8a1667da33c80dd3eb5c54ff7fa3cd3f571433ee2273ea93c12f633a7c17 Jul 24 13:45:15.914137 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.914114 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:15.923938 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.923917 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" Jul 24 13:45:15.930379 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.930284 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421b243e_5770_4c8b_985b_6f58e9e8899e.slice/crio-218f5b2fce0672032877564e72ed9309d0b75000144aa5e58ffdbe3d52edb428 WatchSource:0}: Error finding container 218f5b2fce0672032877564e72ed9309d0b75000144aa5e58ffdbe3d52edb428: Status 404 returned error can't find the container with id 218f5b2fce0672032877564e72ed9309d0b75000144aa5e58ffdbe3d52edb428 Jul 24 13:45:15.932530 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.932275 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" Jul 24 13:45:15.939783 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:15.939756 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:15.941303 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.941273 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302ffe90_feec_4946_b2f8_885f77e67d18.slice/crio-8f04761decbc59940785c1c060c36c73e0ec81894f68e31e8280f6f53ec3b775 WatchSource:0}: Error finding container 8f04761decbc59940785c1c060c36c73e0ec81894f68e31e8280f6f53ec3b775: Status 404 returned error can't find the container with id 8f04761decbc59940785c1c060c36c73e0ec81894f68e31e8280f6f53ec3b775 Jul 24 13:45:15.949692 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:15.949652 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5694c059_720d_48c0_808a_c2ce5c2c3685.slice/crio-6aeb3e29710c626b04cfa900888256189407a6b855691c8bc698626b15c3d92e WatchSource:0}: Error finding container 6aeb3e29710c626b04cfa900888256189407a6b855691c8bc698626b15c3d92e: Status 404 returned error can't find the container with id 6aeb3e29710c626b04cfa900888256189407a6b855691c8bc698626b15c3d92e Jul 24 13:45:16.057202 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.057116 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:45:16.181353 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.181312 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:16.181531 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.181457 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:16.181645 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.181543 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:17.181520711 +0000 UTC m=+3.976863427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:16.282546 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.282499 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:16.282761 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.282689 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:16.282825 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.282776 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:45:17.282750513 +0000 UTC m=+4.078093293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:16.383776 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.383673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:16.383942 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.383880 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:16.383942 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.383904 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:16.383942 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.383914 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:16.384089 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:16.383972 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:45:17.383955907 +0000 UTC m=+4.179298617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:16.606390 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.606276 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:40:15 +0000 UTC" deadline="2028-03-13 07:43:51.872484846 +0000 UTC" Jul 24 13:45:16.606390 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.606319 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14345h58m35.266171084s" Jul 24 13:45:16.760502 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.760459 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"9315f95f4bbb547714e3e1dae6ea3116e7ec235d4408dbf9455f3533087bb04b"} Jul 24 13:45:16.770149 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.770108 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" event={"ID":"5694c059-720d-48c0-808a-c2ce5c2c3685","Type":"ContainerStarted","Data":"6aeb3e29710c626b04cfa900888256189407a6b855691c8bc698626b15c3d92e"} Jul 24 13:45:16.774500 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.774463 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9228l" event={"ID":"f085458a-3b25-43ca-85a4-d79e0ae4a2f8","Type":"ContainerStarted","Data":"65ec5618d3ce0cf39f44c52944052debe2613133eb372c25765c7715835f9037"} Jul 24 13:45:16.778225 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.778186 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerStarted","Data":"0c2c8a1667da33c80dd3eb5c54ff7fa3cd3f571433ee2273ea93c12f633a7c17"} Jul 24 13:45:16.781644 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.781568 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cvnzc" event={"ID":"01d4d8d2-be23-425c-b318-811a3cd71ab8","Type":"ContainerStarted","Data":"078217d21d19b7a199890562f3630c484c7ef6760a35790082be7b0fecb26035"} Jul 24 13:45:16.785099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.785063 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kjr4v" event={"ID":"7a02ec45-abb1-4162-b8e1-b9fd31567f85","Type":"ContainerStarted","Data":"6257948cc8d2cd334f7c895b19705dcd4d3520767db3a574a3c08eb3289b6dd8"} Jul 24 13:45:16.790361 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.790325 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" event={"ID":"302ffe90-feec-4946-b2f8-885f77e67d18","Type":"ContainerStarted","Data":"8f04761decbc59940785c1c060c36c73e0ec81894f68e31e8280f6f53ec3b775"} Jul 24 13:45:16.792009 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.791968 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xpkrh" event={"ID":"421b243e-5770-4c8b-985b-6f58e9e8899e","Type":"ContainerStarted","Data":"218f5b2fce0672032877564e72ed9309d0b75000144aa5e58ffdbe3d52edb428"} Jul 24 13:45:16.803164 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:16.802937 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n9vmm" event={"ID":"be143ea6-d2ee-4709-8fd8-de79390abd1f","Type":"ContainerStarted","Data":"853679da46a76311ba220c77286cead53d87e4bf8a37f69c4ef92a65d9c14ef3"} Jul 24 13:45:17.190772 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.190728 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:17.191028 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.190945 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:17.191112 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.191033 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:19.190994518 +0000 UTC m=+5.986337231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:17.291549 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.291509 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:17.291754 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.291730 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:17.291826 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.291796 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:45:19.291775639 +0000 UTC m=+6.087118353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:17.393196 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.392484 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:17.393196 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.392720 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:17.393196 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.392744 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:17.393196 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.392758 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:17.393196 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.392821 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:45:19.392802018 +0000 UTC m=+6.188144733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:17.607058 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.606934 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:40:15 +0000 UTC" deadline="2028-03-27 02:21:02.125870575 +0000 UTC" Jul 24 13:45:17.607058 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.607001 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14676h35m44.518874285s" Jul 24 13:45:17.746162 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.745129 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:17.746162 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.745485 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:17.746162 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.745918 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:17.746852 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.746608 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:17.746852 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:17.746681 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:17.746852 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:17.746763 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:19.209312 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:19.208620 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:19.209312 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.208831 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:19.209312 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.208995 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:23.208940704 +0000 UTC m=+10.004283416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:19.310699 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:19.310030 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:19.310699 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.310261 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:19.310699 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.310323 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:45:23.310304793 +0000 UTC m=+10.105647504 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:19.410911 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:19.410869 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:19.411189 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.411126 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:19.411189 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.411149 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:19.411189 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.411161 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:19.411330 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.411214 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:45:23.411196587 +0000 UTC m=+10.206539298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:19.746210 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:19.746176 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:19.746400 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.746320 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:19.746715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:19.746690 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:19.746821 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.746791 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:19.746882 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:19.746843 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:19.747030 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:19.746953 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:21.743057 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:21.743021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:21.743438 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:21.743021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:21.743438 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:21.743172 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:21.743438 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:21.743021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:21.743438 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:21.743237 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:21.743438 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:21.743327 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:23.242686 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:23.242644 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:23.243129 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.242798 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:23.243129 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.242868 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:31.242853241 +0000 UTC m=+18.038195951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:23.343538 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:23.343496 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:23.343715 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.343669 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:23.343759 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.343736 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:45:31.343715612 +0000 UTC m=+18.139058324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:23.444675 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:23.444632 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:23.444852 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.444820 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:23.444852 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.444841 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:23.444945 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.444854 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:23.444945 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.444913 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:45:31.444894009 +0000 UTC m=+18.240236731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:23.745386 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:23.745349 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:23.745553 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.745490 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:23.745553 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:23.745544 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:23.745679 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.745606 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:23.745679 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:23.745642 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:23.745771 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:23.745697 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:25.744152 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:25.744102 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:25.744587 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:25.744108 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:25.744587 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:25.744241 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:25.744587 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:25.744355 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:25.744587 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:25.744424 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:25.744587 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:25.744506 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:27.742896 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:27.742860 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:27.743290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:27.742862 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:27.743290 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:27.742998 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:27.743290 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:27.743107 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:27.743290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:27.742865 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:27.743290 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:27.743214 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:29.743506 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:29.743467 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:29.743956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:29.743467 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:29.743956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:29.743622 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:29.743956 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:29.743614 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:29.743956 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:29.743757 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:29.743956 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:29.743852 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:31.300305 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:31.300263 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:31.300820 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.300406 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:31.300820 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.300464 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:47.300448079 +0000 UTC m=+34.095790787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:31.401313 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:31.401267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:31.401472 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.401394 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:31.401472 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.401466 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:45:47.401449337 +0000 UTC m=+34.196792049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:31.502223 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:31.502188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:31.502408 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.502345 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:31.502408 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.502365 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:31.502408 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.502376 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:31.502534 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.502426 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:45:47.502412037 +0000 UTC m=+34.297754746 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:31.743968 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:31.743874 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:31.744139 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.744056 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:31.744458 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:31.744439 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:31.744591 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.744531 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:31.744697 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:31.744671 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:31.744813 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:31.744778 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:33.743519 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:33.743477 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:33.744015 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:33.743521 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:33.744015 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:33.743595 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:33.744015 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:33.743647 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:33.744015 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:33.743700 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:33.744015 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:33.743811 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:35.745236 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.744965 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:35.746352 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.745075 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:35.746352 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.745118 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:35.746352 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:35.745399 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:35.746352 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:35.745684 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:35.746352 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:35.745760 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:35.843470 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.843440 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" event={"ID":"b2b0af06f1bfda6fafb07193c56cc418","Type":"ContainerStarted","Data":"8d107f6350e9567b595c5ae2300929c7a2ccc72a5212b06c0e6e057367f9447c"} Jul 24 13:45:35.845480 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.845457 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"12e43aea93f5eb67b389ecd566664b2ab266b130a1199b026b225bfb9791de1b"} Jul 24 13:45:35.845576 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.845518 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"e8ca70b392b4d61528dd9bc489cb6a0418353b54bcdc25bc3711e2fc91f6033a"} Jul 24 13:45:35.847412 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.847389 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" event={"ID":"5694c059-720d-48c0-808a-c2ce5c2c3685","Type":"ContainerStarted","Data":"739e72c76ce3c256f15226bdb6f8172b2453a9e538374abe7e6d9b7dbfa47194"} Jul 24 13:45:35.848867 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.848847 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xpkrh" event={"ID":"421b243e-5770-4c8b-985b-6f58e9e8899e","Type":"ContainerStarted","Data":"d39a8cfbda1981146173686a9d5fbfc933834e3397b4db64aa78086c9610cd90"} Jul 24 13:45:35.886031 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.883493 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-204.ec2.internal" podStartSLOduration=20.883472169 podStartE2EDuration="20.883472169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:35.861386215 +0000 UTC m=+22.656728936" watchObservedRunningTime="2026-07-24 13:45:35.883472169 +0000 UTC m=+22.678814907" Jul 24 13:45:35.904076 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.903947 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xpkrh" podStartSLOduration=3.608666413 podStartE2EDuration="22.903923947s" totalImagesPullingTime="19.295257534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:35.884025912 +0000 UTC m=+22.679368644" watchObservedRunningTime="2026-07-24 13:45:35.903923947 +0000 UTC m=+22.699266686" Jul 24 13:45:35.904368 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:35.904334 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pjdwk" podStartSLOduration=2.639436445 podStartE2EDuration="21.904321453s" totalImagesPullingTime="19.264885008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:35.903268039 +0000 UTC m=+22.698610770" watchObservedRunningTime="2026-07-24 13:45:35.904321453 +0000 UTC m=+22.699664184" Jul 24 13:45:36.852599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.852565 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9228l" event={"ID":"f085458a-3b25-43ca-85a4-d79e0ae4a2f8","Type":"ContainerStarted","Data":"7f34b149be4602f5ba00ee13630f7970803b93065af95c1ff4eda5eb257dbbe6"} Jul 24 13:45:36.854255 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.854229 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a62685-2897-4199-bc3f-8f4471dbebd6" containerID="0c0e23f3888d9832e96bfa77a487c2bf35a09dd0b454bda52fb333f6a37140da" exitCode=0 Jul 24 13:45:36.854359 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.854282 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerDied","Data":"0c0e23f3888d9832e96bfa77a487c2bf35a09dd0b454bda52fb333f6a37140da"} Jul 24 13:45:36.856088 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.855702 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cvnzc" event={"ID":"01d4d8d2-be23-425c-b318-811a3cd71ab8","Type":"ContainerStarted","Data":"dbe6a2a56adb6be9f61355b07b666cbe4b282a68eb1bc426b091ba3f301403b5"} Jul 24 13:45:36.857123 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.857072 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kjr4v" event={"ID":"7a02ec45-abb1-4162-b8e1-b9fd31567f85","Type":"ContainerStarted","Data":"a919e614e06ea70c01c1cf1d9c1b89e802583d8d5c4d69afb525ad16d9d5ef09"} Jul 24 13:45:36.858531 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.858508 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" event={"ID":"302ffe90-feec-4946-b2f8-885f77e67d18","Type":"ContainerStarted","Data":"79130ecf77c117ef020d5f1dcc1154dbd6fc7d64d8be1a20c0dd8dd2d7b74da3"} Jul 24 13:45:36.859794 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.859777 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n9vmm" event={"ID":"be143ea6-d2ee-4709-8fd8-de79390abd1f","Type":"ContainerStarted","Data":"db78fd855d2882e5e15f45b669f31d2cd061cb566b56378c286a93fcbd502a6f"} Jul 24 13:45:36.861259 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.861238 2555 generic.go:356] "Generic (PLEG): container finished" podID="fde6aac7dcc7c6e378a0f2cd26c09763" containerID="1bfe6bb97e2444d0017b2230865d9abd087cc9933b6abe73d2368f1a82e415fd" exitCode=0 Jul 24 13:45:36.861332 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.861315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" event={"ID":"fde6aac7dcc7c6e378a0f2cd26c09763","Type":"ContainerDied","Data":"1bfe6bb97e2444d0017b2230865d9abd087cc9933b6abe73d2368f1a82e415fd"} Jul 24 13:45:36.864482 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.864458 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"d167b64fbc20a5afb712c692e021127cd781a942f7dae1e3b097660d1a4f12a6"} Jul 24 13:45:36.864581 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.864523 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"f74ebb3ee031e16ad6aca6349b47da862c1e496906455f8288136bf21f5fee60"} Jul 24 13:45:36.864581 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.864562 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"5fe71d103b40f60de6fec1fbe2b9563de0d43430a4dc8b26a723e7cdec37883d"} Jul 24 13:45:36.864662 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.864608 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"182babe5956f0c442956a4b5d2a25213622b3b7577f59b241f9d23c1d47e0e13"} Jul 24 13:45:36.869583 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.869537 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9228l" podStartSLOduration=9.165298189 podStartE2EDuration="23.869522471s" totalImagesPullingTime="14.704224282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:36.869261023 +0000 UTC m=+23.664603754" watchObservedRunningTime="2026-07-24 13:45:36.869522471 +0000 UTC m=+23.664865200" Jul 24 13:45:36.922972 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.922914 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n9vmm" podStartSLOduration=4.616630502 podStartE2EDuration="23.92290291s" totalImagesPullingTime="19.306272408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:36.922675906 +0000 UTC m=+23.718018636" watchObservedRunningTime="2026-07-24 13:45:36.92290291 +0000 UTC m=+23.718245639" Jul 24 13:45:36.925823 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.925796 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:36.926720 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.926356 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:36.954754 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.954705 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kjr4v" podStartSLOduration=5.034300427 podStartE2EDuration="23.954684853s" totalImagesPullingTime="18.920384426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:36.939514419 +0000 UTC m=+23.734857148" watchObservedRunningTime="2026-07-24 13:45:36.954684853 +0000 UTC m=+23.750027632" Jul 24 13:45:36.957872 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:36.955949 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cvnzc" podStartSLOduration=4.673466185 podStartE2EDuration="23.955931476s" totalImagesPullingTime="19.282465291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:36.954620539 +0000 UTC m=+23.749963260" watchObservedRunningTime="2026-07-24 13:45:36.955931476 +0000 UTC m=+23.751274209" Jul 24 13:45:37.232972 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.232945 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 13:45:37.637831 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.637736 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T13:45:37.232964542Z","UUID":"54fcb1bd-ba0a-4d36-95f3-86d80534a48a","Handler":null,"Name":"","Endpoint":""} Jul 24 13:45:37.639625 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.639601 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 13:45:37.639765 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.639644 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 13:45:37.743812 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.743780 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:37.744027 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:37.743919 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:37.744027 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.743795 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:37.744155 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:37.744046 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:37.744155 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.743786 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:37.744155 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:37.744119 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:37.868635 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.868596 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" event={"ID":"fde6aac7dcc7c6e378a0f2cd26c09763","Type":"ContainerStarted","Data":"f538cc5f2e2ae319546bad5b4611c2528da9db7531a72de0dfd1f7ac40959135"} Jul 24 13:45:37.870876 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.870842 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" event={"ID":"302ffe90-feec-4946-b2f8-885f77e67d18","Type":"ContainerStarted","Data":"34ff9a4522086a60380c8324bb9cf5577b1aaa8ec4bc278abd3fb66f45f383d6"} Jul 24 13:45:37.871235 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.871211 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:37.872291 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.872254 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9228l" Jul 24 13:45:37.883933 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.883884 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-204.ec2.internal" podStartSLOduration=22.883867659 podStartE2EDuration="22.883867659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:37.883541596 +0000 UTC m=+24.678884327" watchObservedRunningTime="2026-07-24 13:45:37.883867659 +0000 UTC m=+24.679210391" Jul 24 13:45:37.970637 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:37.970554 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:38.789137 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:38.788899 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:38.875071 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:38.874942 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" event={"ID":"302ffe90-feec-4946-b2f8-885f77e67d18","Type":"ContainerStarted","Data":"7e18c50b5e4ac64c190d540f567d843a37812f8890dd35bd888910c0c2e6fdd9"} Jul 24 13:45:38.875585 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:38.875204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:38.881852 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:38.879475 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"8c4907cce85a3b4c8c762f7f297533f9fb14d0e7a3a253e3a3a013be96ab0088"} Jul 24 13:45:38.895713 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:38.895651 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fpq4p" podStartSLOduration=3.038063855 podStartE2EDuration="24.895636878s" totalImagesPullingTime="21.857573023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:38.893753643 +0000 UTC m=+25.689096375" watchObservedRunningTime="2026-07-24 13:45:38.895636878 +0000 UTC m=+25.690979607" Jul 24 13:45:38.977457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:38.977428 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:39.743427 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:39.743390 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:39.743672 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:39.743392 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:39.743672 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:39.743536 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:39.743672 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:39.743567 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:39.743836 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:39.743711 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:39.743836 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:39.743821 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:41.744096 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.743833 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:41.744892 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.743834 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:41.744892 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:41.744147 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:41.744892 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.743834 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:41.744892 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:41.744219 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:41.744892 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:41.744266 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:41.890061 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.890015 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" event={"ID":"f5a273ff-bb8b-4756-9315-0db5130d47bc","Type":"ContainerStarted","Data":"cf0e59845f8b799da55b411168ddc37945efa001d1685168a6e210bf1b8fa28d"} Jul 24 13:45:41.890298 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.890279 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:41.891835 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.891802 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a62685-2897-4199-bc3f-8f4471dbebd6" containerID="d9df43bd3979c76f063bbc6f04df95296d33d18cbc0fd7918c336cd3aa2206b5" exitCode=0 Jul 24 13:45:41.891932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.891859 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerDied","Data":"d9df43bd3979c76f063bbc6f04df95296d33d18cbc0fd7918c336cd3aa2206b5"} Jul 24 13:45:41.907678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.907651 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:41.919183 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:41.917474 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" podStartSLOduration=8.310638546 podStartE2EDuration="27.917453947s" totalImagesPullingTime="19.606815401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:41.916685732 +0000 UTC m=+28.712028466" watchObservedRunningTime="2026-07-24 13:45:41.917453947 +0000 UTC m=+28.712796678" Jul 24 13:45:42.896309 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:42.896277 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a62685-2897-4199-bc3f-8f4471dbebd6" containerID="b041d6d10881648f8208b7b61321fa62461878e18e108f7edc12d866396f339e" exitCode=0 Jul 24 13:45:42.896742 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:42.896362 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerDied","Data":"b041d6d10881648f8208b7b61321fa62461878e18e108f7edc12d866396f339e"} Jul 24 13:45:42.896804 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:42.896772 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:42.896865 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:42.896851 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:42.913183 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:42.913151 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:45:43.201365 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.201334 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:43.264376 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.264348 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:43.363565 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.363535 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:43.407581 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.407322 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wsr7q"] Jul 24 13:45:43.407581 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.407457 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:43.407581 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:43.407575 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:43.410698 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.410639 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p9p9n"] Jul 24 13:45:43.410869 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.410767 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:43.410961 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:43.410875 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:43.411236 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.411214 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ql85p"] Jul 24 13:45:43.411329 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.411318 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:43.411441 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:43.411415 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:43.463867 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.463780 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:43.901121 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.900923 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a62685-2897-4199-bc3f-8f4471dbebd6" containerID="2c1b4cffbb5e454c6acb6b5eb2c46d194be65d934dbee9691821fde7163f25e3" exitCode=0 Jul 24 13:45:43.901529 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:43.901007 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerDied","Data":"2c1b4cffbb5e454c6acb6b5eb2c46d194be65d934dbee9691821fde7163f25e3"} Jul 24 13:45:44.290406 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:44.290321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:44.390809 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:44.390771 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:44.743529 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:44.743498 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:44.743729 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:44.743648 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:44.743729 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:44.743690 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:44.743833 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:44.743757 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:45.743444 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:45.743401 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:45.744064 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:45.743554 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:46.743554 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:46.743509 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:46.744189 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:46.743655 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wsr7q" podUID="2ce8fd08-5295-4821-9d6f-c7bed4fc260f" Jul 24 13:45:46.744189 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:46.743708 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:46.744189 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:46.743836 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p9p9n" podUID="9aaf33c3-bc80-4060-88f7-09857fc9e81b" Jul 24 13:45:47.315246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:47.315208 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:47.315438 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.315388 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:47.315497 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.315454 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:46:19.315438018 +0000 UTC m=+66.110780744 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:45:47.416473 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:47.416442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:47.416672 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.416574 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:47.416672 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.416640 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret podName:2ce8fd08-5295-4821-9d6f-c7bed4fc260f nodeName:}" failed. No retries permitted until 2026-07-24 13:46:19.416623185 +0000 UTC m=+66.211965896 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret") pod "global-pull-secret-syncer-wsr7q" (UID: "2ce8fd08-5295-4821-9d6f-c7bed4fc260f") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:45:47.517832 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:47.517792 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:47.518023 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.518007 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:45:47.518066 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.518032 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:45:47.518066 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.518045 2555 projected.go:196] Error preparing data for projected volume kube-api-access-g492p for pod openshift-network-diagnostics/network-check-target-p9p9n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:47.518131 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.518106 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p podName:9aaf33c3-bc80-4060-88f7-09857fc9e81b nodeName:}" failed. No retries permitted until 2026-07-24 13:46:19.518087833 +0000 UTC m=+66.313430542 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g492p" (UniqueName: "kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p") pod "network-check-target-p9p9n" (UID: "9aaf33c3-bc80-4060-88f7-09857fc9e81b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:45:47.540326 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:47.540276 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:47.594198 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:47.594121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:47.743290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:47.743238 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:47.743442 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:47.743423 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:45:48.015489 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.015459 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-204.ec2.internal" event="NodeReady" Jul 24 13:45:48.015971 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.015586 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 13:45:48.062537 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.062502 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xknnz"] Jul 24 13:45:48.069323 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.069294 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sc55j"] Jul 24 13:45:48.069490 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.069426 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.071804 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.071785 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gc8ks\"" Jul 24 13:45:48.072014 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.071968 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 13:45:48.072124 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.072061 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 13:45:48.072277 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.072264 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 13:45:48.075652 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.075615 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xknnz"] Jul 24 13:45:48.075759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.075726 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.078188 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.078163 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sc55j"] Jul 24 13:45:48.078302 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.078206 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 13:45:48.078302 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.078286 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7f5lp\"" Jul 24 13:45:48.078599 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.078581 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 13:45:48.221733 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.221639 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-config-volume\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.221733 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.221685 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.221733 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.221730 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-tmp-dir\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.222019 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.221782 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv2zp\" (UniqueName: \"kubernetes.io/projected/1c1a5e07-f00e-4480-b0f1-f2fa6624461a-kube-api-access-cv2zp\") pod \"ingress-canary-xknnz\" (UID: \"1c1a5e07-f00e-4480-b0f1-f2fa6624461a\") " pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.222019 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.221831 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scztj\" (UniqueName: \"kubernetes.io/projected/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-kube-api-access-scztj\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.222019 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.221925 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c1a5e07-f00e-4480-b0f1-f2fa6624461a-cert\") pod \"ingress-canary-xknnz\" (UID: \"1c1a5e07-f00e-4480-b0f1-f2fa6624461a\") " pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.231014 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.230950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:48.323123 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323095 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c1a5e07-f00e-4480-b0f1-f2fa6624461a-cert\") pod \"ingress-canary-xknnz\" (UID: \"1c1a5e07-f00e-4480-b0f1-f2fa6624461a\") " pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.323123 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323127 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-config-volume\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.323405 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.323405 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:48.323245 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:45:48.323405 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-tmp-dir\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.323405 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:48.323296 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:48.823283556 +0000 UTC m=+35.618626288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:45:48.323405 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv2zp\" (UniqueName: \"kubernetes.io/projected/1c1a5e07-f00e-4480-b0f1-f2fa6624461a-kube-api-access-cv2zp\") pod \"ingress-canary-xknnz\" (UID: \"1c1a5e07-f00e-4480-b0f1-f2fa6624461a\") " pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.323405 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323379 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scztj\" (UniqueName: \"kubernetes.io/projected/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-kube-api-access-scztj\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.323710 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323582 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-tmp-dir\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.323834 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.323804 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-config-volume\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.327564 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.327532 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1c1a5e07-f00e-4480-b0f1-f2fa6624461a-cert\") pod \"ingress-canary-xknnz\" (UID: \"1c1a5e07-f00e-4480-b0f1-f2fa6624461a\") " pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.337911 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.337883 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scztj\" (UniqueName: \"kubernetes.io/projected/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-kube-api-access-scztj\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.338053 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.337955 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv2zp\" (UniqueName: \"kubernetes.io/projected/1c1a5e07-f00e-4480-b0f1-f2fa6624461a-kube-api-access-cv2zp\") pod \"ingress-canary-xknnz\" (UID: \"1c1a5e07-f00e-4480-b0f1-f2fa6624461a\") " pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.380042 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.380005 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xknnz" Jul 24 13:45:48.558617 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.558426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xknnz"] Jul 24 13:45:48.562039 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:45:48.562014 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c1a5e07_f00e_4480_b0f1_f2fa6624461a.slice/crio-b7e56c2eba43a98cb8dfa347a570afa26abd4ee16107f12645dab1546af11b12 WatchSource:0}: Error finding container b7e56c2eba43a98cb8dfa347a570afa26abd4ee16107f12645dab1546af11b12: Status 404 returned error can't find the container with id b7e56c2eba43a98cb8dfa347a570afa26abd4ee16107f12645dab1546af11b12 Jul 24 13:45:48.743452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.743345 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:45:48.743452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.743387 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:45:48.746292 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.746269 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:45:48.746521 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.746506 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h5k24\"" Jul 24 13:45:48.746587 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.746528 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:45:48.746746 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.746731 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:45:48.827569 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.827529 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:48.827753 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:48.827673 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:45:48.827753 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:48.827731 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:49.82771593 +0000 UTC m=+36.623058639 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:45:48.927887 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.927854 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a62685-2897-4199-bc3f-8f4471dbebd6" containerID="7a9dc79619a30a25b4c1f8b43aedaa05de66a6e88889b27c283d88ae426be9ea" exitCode=0 Jul 24 13:45:48.928134 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.927935 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerDied","Data":"7a9dc79619a30a25b4c1f8b43aedaa05de66a6e88889b27c283d88ae426be9ea"} Jul 24 13:45:48.929139 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:48.929113 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xknnz" event={"ID":"1c1a5e07-f00e-4480-b0f1-f2fa6624461a","Type":"ContainerStarted","Data":"b7e56c2eba43a98cb8dfa347a570afa26abd4ee16107f12645dab1546af11b12"} Jul 24 13:45:49.743438 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:49.743407 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:45:49.746453 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:49.746426 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:45:49.746589 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:49.746436 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qbf75\"" Jul 24 13:45:49.836153 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:49.836108 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:49.836380 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:49.836266 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:45:49.836380 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:49.836327 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:51.836308015 +0000 UTC m=+38.631650741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:45:49.934810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:49.934775 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a62685-2897-4199-bc3f-8f4471dbebd6" containerID="773808f88d642e902e3da11a5496e0d56a2e83e24831f9f79e813183a4b77241" exitCode=0 Jul 24 13:45:49.934810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:49.934815 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerDied","Data":"773808f88d642e902e3da11a5496e0d56a2e83e24831f9f79e813183a4b77241"} Jul 24 13:45:50.940116 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:50.940081 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" event={"ID":"f7a62685-2897-4199-bc3f-8f4471dbebd6","Type":"ContainerStarted","Data":"e1b4fc741877a258305ee1b97795c32489a414c934e629badb8588108e4a24be"} Jul 24 13:45:50.964093 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:50.964040 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kb9dl" podStartSLOduration=8.603692265 podStartE2EDuration="37.964021537s" totalImagesPullingTime="29.360329272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:50.962083412 +0000 UTC m=+37.757426143" watchObservedRunningTime="2026-07-24 13:45:50.964021537 +0000 UTC m=+37.759364266" Jul 24 13:45:51.084330 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:51.084294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:51.187276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:51.187247 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:51.852559 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:51.852326 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:51.852770 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:51.852489 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:45:51.852770 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:51.852667 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:45:55.852648042 +0000 UTC m=+42.647990749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:45:51.943710 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:51.943652 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xknnz" event={"ID":"1c1a5e07-f00e-4480-b0f1-f2fa6624461a","Type":"ContainerStarted","Data":"f15d4e6fe4f8931a0b0fe2a8e551d6ac589476b6a1f09d536811c6ebb1d0d1c0"} Jul 24 13:45:51.959441 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:51.959392 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xknnz" podStartSLOduration=1.400479877 podStartE2EDuration="3.959376489s" totalImagesPullingTime="2.558896612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:45:51.958520453 +0000 UTC m=+38.753863183" watchObservedRunningTime="2026-07-24 13:45:51.959376489 +0000 UTC m=+38.754719219" Jul 24 13:45:53.642458 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:53.642430 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:53.743030 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:53.742970 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:54.145388 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:54.145360 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:55.825846 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:55.825803 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:55.883338 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:55.883302 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:45:55.883487 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:55.883468 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:45:55.883540 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:45:55.883529 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:46:03.883514029 +0000 UTC m=+50.678856737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:45:59.880604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:59.880567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:45:59.935140 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:45:59.935107 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:00.333235 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:00.333204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:00.435043 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:00.435012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:00.492013 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:00.491968 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:03.944072 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:03.944026 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:46:03.944461 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:03.944183 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:46:03.944461 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:03.944259 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:46:19.944242815 +0000 UTC m=+66.739585522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:46:14.916242 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:14.916213 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b8jcw" Jul 24 13:46:15.066913 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:15.066876 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:15.119606 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:15.119583 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:19.353711 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.353672 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:46:19.356998 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.356953 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:46:19.363976 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:19.363952 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 24 13:46:19.364123 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:19.364043 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:47:23.364026067 +0000 UTC m=+130.159368775 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : secret "metrics-daemon-secret" not found Jul 24 13:46:19.454848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.454809 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:46:19.457939 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.457922 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:46:19.469150 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.469119 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2ce8fd08-5295-4821-9d6f-c7bed4fc260f-original-pull-secret\") pod \"global-pull-secret-syncer-wsr7q\" (UID: \"2ce8fd08-5295-4821-9d6f-c7bed4fc260f\") " pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:46:19.555736 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.555700 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:46:19.558826 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.558807 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:46:19.568921 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.568895 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:46:19.579609 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.579583 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g492p\" (UniqueName: \"kubernetes.io/projected/9aaf33c3-bc80-4060-88f7-09857fc9e81b-kube-api-access-g492p\") pod \"network-check-target-p9p9n\" (UID: \"9aaf33c3-bc80-4060-88f7-09857fc9e81b\") " pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:46:19.656600 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.656516 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h5k24\"" Jul 24 13:46:19.658532 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.658512 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wsr7q" Jul 24 13:46:19.663828 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.663803 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:46:19.823426 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.823391 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wsr7q"] Jul 24 13:46:19.826127 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:46:19.826095 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce8fd08_5295_4821_9d6f_c7bed4fc260f.slice/crio-bedb80484a26eb69f1209f94a57dbe0e6dcf40521c8f860c4d459da3cb8b2790 WatchSource:0}: Error finding container bedb80484a26eb69f1209f94a57dbe0e6dcf40521c8f860c4d459da3cb8b2790: Status 404 returned error can't find the container with id bedb80484a26eb69f1209f94a57dbe0e6dcf40521c8f860c4d459da3cb8b2790 Jul 24 13:46:19.847044 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.847015 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p9p9n"] Jul 24 13:46:19.850870 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:46:19.850845 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aaf33c3_bc80_4060_88f7_09857fc9e81b.slice/crio-a4a5bcf18f0ff7e5a7d532dcc5d46a2f874de0b8889a9faad281a64683ab3b79 WatchSource:0}: Error finding container a4a5bcf18f0ff7e5a7d532dcc5d46a2f874de0b8889a9faad281a64683ab3b79: Status 404 returned error can't find the container with id a4a5bcf18f0ff7e5a7d532dcc5d46a2f874de0b8889a9faad281a64683ab3b79 Jul 24 13:46:19.957725 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:19.957634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:46:19.957868 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:19.957741 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:46:19.957868 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:19.957797 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:46:51.957783105 +0000 UTC m=+98.753125812 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:46:20.009717 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:20.009680 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p9p9n" event={"ID":"9aaf33c3-bc80-4060-88f7-09857fc9e81b","Type":"ContainerStarted","Data":"a4a5bcf18f0ff7e5a7d532dcc5d46a2f874de0b8889a9faad281a64683ab3b79"} Jul 24 13:46:20.010768 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:20.010745 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wsr7q" event={"ID":"2ce8fd08-5295-4821-9d6f-c7bed4fc260f","Type":"ContainerStarted","Data":"bedb80484a26eb69f1209f94a57dbe0e6dcf40521c8f860c4d459da3cb8b2790"} Jul 24 13:46:21.833906 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.833867 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26"] Jul 24 13:46:21.837254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.837229 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:21.839949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.839919 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 24 13:46:21.839949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.839938 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 24 13:46:21.840226 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.840208 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 24 13:46:21.841340 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.841093 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-59nnp\"" Jul 24 13:46:21.842389 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.842349 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 24 13:46:21.843635 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.843612 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n"] Jul 24 13:46:21.846222 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.846201 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26"] Jul 24 13:46:21.846325 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.846312 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.849272 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.849243 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 24 13:46:21.858678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.858536 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n"] Jul 24 13:46:21.883310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.883271 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t759l\" (UniqueName: \"kubernetes.io/projected/ca948322-6874-44ce-b516-023505d2eaa4-kube-api-access-t759l\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.883310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.883310 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ca948322-6874-44ce-b516-023505d2eaa4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.883539 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.883349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17b72dfb-6d52-4652-b8cd-ed0a9cff7676-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-647f9866-plq26\" (UID: \"17b72dfb-6d52-4652-b8cd-ed0a9cff7676\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:21.883539 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.883377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssg66\" (UniqueName: \"kubernetes.io/projected/17b72dfb-6d52-4652-b8cd-ed0a9cff7676-kube-api-access-ssg66\") pod \"managed-serviceaccount-addon-agent-647f9866-plq26\" (UID: \"17b72dfb-6d52-4652-b8cd-ed0a9cff7676\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:21.883539 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.883419 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca948322-6874-44ce-b516-023505d2eaa4-tmp\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.984108 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.984076 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca948322-6874-44ce-b516-023505d2eaa4-tmp\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.984286 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.984135 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t759l\" (UniqueName: \"kubernetes.io/projected/ca948322-6874-44ce-b516-023505d2eaa4-kube-api-access-t759l\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.984286 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.984153 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ca948322-6874-44ce-b516-023505d2eaa4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.984286 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.984179 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17b72dfb-6d52-4652-b8cd-ed0a9cff7676-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-647f9866-plq26\" (UID: \"17b72dfb-6d52-4652-b8cd-ed0a9cff7676\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:21.984286 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.984197 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg66\" (UniqueName: \"kubernetes.io/projected/17b72dfb-6d52-4652-b8cd-ed0a9cff7676-kube-api-access-ssg66\") pod \"managed-serviceaccount-addon-agent-647f9866-plq26\" (UID: \"17b72dfb-6d52-4652-b8cd-ed0a9cff7676\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:21.984740 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.984721 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca948322-6874-44ce-b516-023505d2eaa4-tmp\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.987993 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.987956 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17b72dfb-6d52-4652-b8cd-ed0a9cff7676-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-647f9866-plq26\" (UID: \"17b72dfb-6d52-4652-b8cd-ed0a9cff7676\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:21.988141 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.988113 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ca948322-6874-44ce-b516-023505d2eaa4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.993629 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.993600 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t759l\" (UniqueName: \"kubernetes.io/projected/ca948322-6874-44ce-b516-023505d2eaa4-kube-api-access-t759l\") pod \"klusterlet-addon-workmgr-7859878df8-p999n\" (UID: \"ca948322-6874-44ce-b516-023505d2eaa4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:21.994200 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:21.994154 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssg66\" (UniqueName: \"kubernetes.io/projected/17b72dfb-6d52-4652-b8cd-ed0a9cff7676-kube-api-access-ssg66\") pod \"managed-serviceaccount-addon-agent-647f9866-plq26\" (UID: \"17b72dfb-6d52-4652-b8cd-ed0a9cff7676\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:22.154177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:22.154081 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" Jul 24 13:46:22.190965 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:22.190928 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:22.321552 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:22.321518 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:22.418472 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:22.418390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:24.373327 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:24.373288 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26"] Jul 24 13:46:24.375642 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:46:24.375617 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17b72dfb_6d52_4652_b8cd_ed0a9cff7676.slice/crio-3c61962c67530a9fa7df101ca0974254e04386c6cf3b0e86a0c8dc130fd0bf73 WatchSource:0}: Error finding container 3c61962c67530a9fa7df101ca0974254e04386c6cf3b0e86a0c8dc130fd0bf73: Status 404 returned error can't find the container with id 3c61962c67530a9fa7df101ca0974254e04386c6cf3b0e86a0c8dc130fd0bf73 Jul 24 13:46:24.387434 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:24.387403 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n"] Jul 24 13:46:24.390338 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:46:24.390311 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca948322_6874_44ce_b516_023505d2eaa4.slice/crio-c75a90d60b48688950da2a3c561974f99d5a3dd0c27e55400176b4fe8f84fb40 WatchSource:0}: Error finding container c75a90d60b48688950da2a3c561974f99d5a3dd0c27e55400176b4fe8f84fb40: Status 404 returned error can't find the container with id c75a90d60b48688950da2a3c561974f99d5a3dd0c27e55400176b4fe8f84fb40 Jul 24 13:46:25.025668 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.025629 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wsr7q" event={"ID":"2ce8fd08-5295-4821-9d6f-c7bed4fc260f","Type":"ContainerStarted","Data":"d33efa8662d042906477f2dbbd21070a89a18715f28abb0247076ad4c5277731"} Jul 24 13:46:25.027229 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.027204 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p9p9n" event={"ID":"9aaf33c3-bc80-4060-88f7-09857fc9e81b","Type":"ContainerStarted","Data":"e6269426c24291123f35b9cfc9d2af009978af5d50b6fc173d721b56c0bb2cf1"} Jul 24 13:46:25.027351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.027277 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:46:25.028249 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.028220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" event={"ID":"ca948322-6874-44ce-b516-023505d2eaa4","Type":"ContainerStarted","Data":"c75a90d60b48688950da2a3c561974f99d5a3dd0c27e55400176b4fe8f84fb40"} Jul 24 13:46:25.029101 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.029083 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" event={"ID":"17b72dfb-6d52-4652-b8cd-ed0a9cff7676","Type":"ContainerStarted","Data":"3c61962c67530a9fa7df101ca0974254e04386c6cf3b0e86a0c8dc130fd0bf73"} Jul 24 13:46:25.041690 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.041637 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wsr7q" podStartSLOduration=66.633180094 podStartE2EDuration="1m11.041622184s" totalImagesPullingTime="4.40844209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:46:25.041356254 +0000 UTC m=+71.836698985" watchObservedRunningTime="2026-07-24 13:46:25.041622184 +0000 UTC m=+71.836964892" Jul 24 13:46:25.056935 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:25.056881 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p9p9n" podStartSLOduration=67.681467942 podStartE2EDuration="1m12.056863817s" totalImagesPullingTime="4.375395875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:46:25.056327528 +0000 UTC m=+71.851670258" watchObservedRunningTime="2026-07-24 13:46:25.056863817 +0000 UTC m=+71.852206547" Jul 24 13:46:29.042874 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:29.042837 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" event={"ID":"ca948322-6874-44ce-b516-023505d2eaa4","Type":"ContainerStarted","Data":"6932040b2a464d0bb3bbed2085d1639150aac9e38085a9c73da03f90cb5ec576"} Jul 24 13:46:29.043356 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:29.043007 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:29.044482 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:29.044454 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" event={"ID":"17b72dfb-6d52-4652-b8cd-ed0a9cff7676","Type":"ContainerStarted","Data":"f95c91f29ef7df8950d5724de6d9618653ab923fb451224c908179e157cbee8d"} Jul 24 13:46:29.044835 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:29.044823 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:46:29.059545 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:29.059495 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" podStartSLOduration=3.984625924 podStartE2EDuration="8.05948165s" totalImagesPullingTime="4.074855726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:46:29.059176214 +0000 UTC m=+75.854518959" watchObservedRunningTime="2026-07-24 13:46:29.05948165 +0000 UTC m=+75.854824379" Jul 24 13:46:29.074465 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:29.074412 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" podStartSLOduration=3.994633082 podStartE2EDuration="8.074395745s" totalImagesPullingTime="4.079762663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:46:29.074102571 +0000 UTC m=+75.869445300" watchObservedRunningTime="2026-07-24 13:46:29.074395745 +0000 UTC m=+75.869738475" Jul 24 13:46:51.992185 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:51.992046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:46:51.992185 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:51.992161 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:46:51.992692 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:46:51.992222 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:47:55.992204781 +0000 UTC m=+162.787547503 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:46:54.571557 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:54.571524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:54.671965 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:54.671934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:46:56.035616 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:46:56.035588 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p9p9n" Jul 24 13:47:23.414586 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:47:23.414542 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:47:23.415083 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:47:23.414696 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 24 13:47:23.415083 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:47:23.414779 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:25.414762819 +0000 UTC m=+252.210105531 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : secret "metrics-daemon-secret" not found Jul 24 13:47:51.086600 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:47:51.086554 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-sc55j" podUID="4949ccfd-f2b4-44b4-9e00-2228bb2ad367" Jul 24 13:47:51.251921 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:47:51.251888 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sc55j" Jul 24 13:47:52.753008 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:47:52.752930 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:47:56.046147 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:47:56.046107 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:47:56.046561 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:47:56.046229 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 13:47:56.046561 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:47:56.046305 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls podName:4949ccfd-f2b4-44b4-9e00-2228bb2ad367 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:58.046287807 +0000 UTC m=+284.841630521 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls") pod "dns-default-sc55j" (UID: "4949ccfd-f2b4-44b4-9e00-2228bb2ad367") : secret "dns-default-metrics-tls" not found Jul 24 13:48:03.745002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:03.744958 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:48:04.779898 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:04.779864 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:48:29.043659 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.043610 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" podUID="ca948322-6874-44ce-b516-023505d2eaa4" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jul 24 13:48:29.347208 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.347119 2555 generic.go:356] "Generic (PLEG): container finished" podID="ca948322-6874-44ce-b516-023505d2eaa4" containerID="6932040b2a464d0bb3bbed2085d1639150aac9e38085a9c73da03f90cb5ec576" exitCode=1 Jul 24 13:48:29.347363 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.347207 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" event={"ID":"ca948322-6874-44ce-b516-023505d2eaa4","Type":"ContainerDied","Data":"6932040b2a464d0bb3bbed2085d1639150aac9e38085a9c73da03f90cb5ec576"} Jul 24 13:48:29.347597 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.347571 2555 scope.go:117] "RemoveContainer" containerID="6932040b2a464d0bb3bbed2085d1639150aac9e38085a9c73da03f90cb5ec576" Jul 24 13:48:29.348623 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.348603 2555 generic.go:356] "Generic (PLEG): container finished" podID="17b72dfb-6d52-4652-b8cd-ed0a9cff7676" containerID="f95c91f29ef7df8950d5724de6d9618653ab923fb451224c908179e157cbee8d" exitCode=255 Jul 24 13:48:29.348690 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.348632 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" event={"ID":"17b72dfb-6d52-4652-b8cd-ed0a9cff7676","Type":"ContainerDied","Data":"f95c91f29ef7df8950d5724de6d9618653ab923fb451224c908179e157cbee8d"} Jul 24 13:48:29.354561 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:29.354540 2555 scope.go:117] "RemoveContainer" containerID="f95c91f29ef7df8950d5724de6d9618653ab923fb451224c908179e157cbee8d" Jul 24 13:48:30.353291 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:30.353255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" event={"ID":"ca948322-6874-44ce-b516-023505d2eaa4","Type":"ContainerStarted","Data":"84b4c40b013741ccbb2dd2d109bfd4ebba474590611c89caf58f59149da0e680"} Jul 24 13:48:30.353758 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:30.353509 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:48:30.354235 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:30.354215 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7859878df8-p999n" Jul 24 13:48:30.355048 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:30.355028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-647f9866-plq26" event={"ID":"17b72dfb-6d52-4652-b8cd-ed0a9cff7676","Type":"ContainerStarted","Data":"8bc8df24fb96062e385f6f0b08f77aced53e19566adc26a24e26eef229579459"} Jul 24 13:48:48.975716 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:48:48.975682 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:19.170674 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.170642 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh"] Jul 24 13:49:19.172915 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.172894 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.175596 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.175580 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lhbmn\"" Jul 24 13:49:19.175695 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.175675 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 24 13:49:19.176776 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.176759 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:49:19.176858 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.176844 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 24 13:49:19.189034 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.189008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh"] Jul 24 13:49:19.280746 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.280707 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb"] Jul 24 13:49:19.282802 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.282777 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" Jul 24 13:49:19.289259 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.289235 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bv27m\"" Jul 24 13:49:19.316111 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.316081 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.316303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.316145 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxzft\" (UniqueName: \"kubernetes.io/projected/d4d4123c-fd9e-4f02-934c-5295f09a6bca-kube-api-access-bxzft\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.319298 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.319273 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb"] Jul 24 13:49:19.416796 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.416758 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxzft\" (UniqueName: \"kubernetes.io/projected/d4d4123c-fd9e-4f02-934c-5295f09a6bca-kube-api-access-bxzft\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.416796 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.416797 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7gm7\" (UniqueName: \"kubernetes.io/projected/072b80ef-e0d3-4022-845c-44606a63d7c4-kube-api-access-t7gm7\") pod \"network-check-source-7d88cb55d8-nzjkb\" (UID: \"072b80ef-e0d3-4022-845c-44606a63d7c4\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" Jul 24 13:49:19.417104 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.416842 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.417104 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:19.416930 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:49:19.417104 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:19.417014 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls podName:d4d4123c-fd9e-4f02-934c-5295f09a6bca nodeName:}" failed. No retries permitted until 2026-07-24 13:49:19.91697458 +0000 UTC m=+246.712317290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls") pod "cluster-samples-operator-6748489656-s6rhh" (UID: "d4d4123c-fd9e-4f02-934c-5295f09a6bca") : secret "samples-operator-tls" not found Jul 24 13:49:19.427827 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.427769 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxzft\" (UniqueName: \"kubernetes.io/projected/d4d4123c-fd9e-4f02-934c-5295f09a6bca-kube-api-access-bxzft\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.503114 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.503081 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:19.518173 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.518134 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7gm7\" (UniqueName: \"kubernetes.io/projected/072b80ef-e0d3-4022-845c-44606a63d7c4-kube-api-access-t7gm7\") pod \"network-check-source-7d88cb55d8-nzjkb\" (UID: \"072b80ef-e0d3-4022-845c-44606a63d7c4\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" Jul 24 13:49:19.527019 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.526968 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7gm7\" (UniqueName: \"kubernetes.io/projected/072b80ef-e0d3-4022-845c-44606a63d7c4-kube-api-access-t7gm7\") pod \"network-check-source-7d88cb55d8-nzjkb\" (UID: \"072b80ef-e0d3-4022-845c-44606a63d7c4\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" Jul 24 13:49:19.591136 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.591098 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" Jul 24 13:49:19.716452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.716371 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb"] Jul 24 13:49:19.719741 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:49:19.719712 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072b80ef_e0d3_4022_845c_44606a63d7c4.slice/crio-169ff75c1f546a5c1c7a5a3ec64d0a49053ce2f2ea3b76c6927b8c81cde8ec8b WatchSource:0}: Error finding container 169ff75c1f546a5c1c7a5a3ec64d0a49053ce2f2ea3b76c6927b8c81cde8ec8b: Status 404 returned error can't find the container with id 169ff75c1f546a5c1c7a5a3ec64d0a49053ce2f2ea3b76c6927b8c81cde8ec8b Jul 24 13:49:19.920046 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:19.919961 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:19.920294 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:19.920158 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:49:19.920294 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:19.920240 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls podName:d4d4123c-fd9e-4f02-934c-5295f09a6bca nodeName:}" failed. No retries permitted until 2026-07-24 13:49:20.920218905 +0000 UTC m=+247.715561619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls") pod "cluster-samples-operator-6748489656-s6rhh" (UID: "d4d4123c-fd9e-4f02-934c-5295f09a6bca") : secret "samples-operator-tls" not found Jul 24 13:49:20.498032 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:20.497976 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" event={"ID":"072b80ef-e0d3-4022-845c-44606a63d7c4","Type":"ContainerStarted","Data":"8ac59f75d6903fe9cfbd64cb06f76a6b442db0a20ea6492bd1714e9a49866044"} Jul 24 13:49:20.498500 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:20.498043 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" event={"ID":"072b80ef-e0d3-4022-845c-44606a63d7c4","Type":"ContainerStarted","Data":"169ff75c1f546a5c1c7a5a3ec64d0a49053ce2f2ea3b76c6927b8c81cde8ec8b"} Jul 24 13:49:20.514829 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:20.514775 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nzjkb" podStartSLOduration=1.514757371 podStartE2EDuration="1.514757371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:49:20.513119003 +0000 UTC m=+247.308461745" watchObservedRunningTime="2026-07-24 13:49:20.514757371 +0000 UTC m=+247.310100139" Jul 24 13:49:20.928903 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:20.928863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:20.929091 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:20.929007 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:49:20.929091 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:20.929066 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls podName:d4d4123c-fd9e-4f02-934c-5295f09a6bca nodeName:}" failed. No retries permitted until 2026-07-24 13:49:22.929051492 +0000 UTC m=+249.724394203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls") pod "cluster-samples-operator-6748489656-s6rhh" (UID: "d4d4123c-fd9e-4f02-934c-5295f09a6bca") : secret "samples-operator-tls" not found Jul 24 13:49:20.979683 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:20.979650 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:22.089855 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:22.089818 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:22.148116 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:22.148088 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:22.943434 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:22.943404 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:22.943622 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:22.943550 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:49:22.943622 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:22.943616 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls podName:d4d4123c-fd9e-4f02-934c-5295f09a6bca nodeName:}" failed. No retries permitted until 2026-07-24 13:49:26.943599851 +0000 UTC m=+253.738942558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls") pod "cluster-samples-operator-6748489656-s6rhh" (UID: "d4d4123c-fd9e-4f02-934c-5295f09a6bca") : secret "samples-operator-tls" not found Jul 24 13:49:24.920922 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:24.920887 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.021781 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.021748 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.123505 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.123472 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.220758 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.220683 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.347084 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.347054 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bqc9b"] Jul 24 13:49:25.349708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.349688 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.353540 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.353513 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 13:49:25.353664 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.353628 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 13:49:25.354079 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.354064 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 13:49:25.354836 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.354824 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dqt9s\"" Jul 24 13:49:25.356300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.355933 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 13:49:25.365907 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.365869 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqc9b"] Jul 24 13:49:25.387854 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.387820 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.455189 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.455157 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.460229 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.460205 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-685w2\" (UniqueName: \"kubernetes.io/projected/62578128-8b67-4150-aeb4-12efb712bfe1-kube-api-access-685w2\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.460355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.460243 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62578128-8b67-4150-aeb4-12efb712bfe1-crio-socket\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.460355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.460275 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.460464 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.460354 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:49:25.460464 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.460400 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.460464 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.460432 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62578128-8b67-4150-aeb4-12efb712bfe1-data-volume\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.460576 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:25.460484 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 24 13:49:25.460576 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:25.460539 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs podName:3c5e9d55-41ee-4f12-ae25-74947f9bc6f5 nodeName:}" failed. No retries permitted until 2026-07-24 13:51:27.460522623 +0000 UTC m=+374.255865331 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs") pod "network-metrics-daemon-ql85p" (UID: "3c5e9d55-41ee-4f12-ae25-74947f9bc6f5") : secret "metrics-daemon-secret" not found Jul 24 13:49:25.561699 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.561667 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-685w2\" (UniqueName: \"kubernetes.io/projected/62578128-8b67-4150-aeb4-12efb712bfe1-kube-api-access-685w2\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.561851 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.561710 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62578128-8b67-4150-aeb4-12efb712bfe1-crio-socket\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.561851 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.561739 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.561851 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.561823 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62578128-8b67-4150-aeb4-12efb712bfe1-crio-socket\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.561851 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:25.561837 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:49:25.562009 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.561882 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.562009 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:25.561902 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:26.061883236 +0000 UTC m=+252.857225946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : secret "insights-runtime-extractor-tls" not found Jul 24 13:49:25.562009 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:25.561929 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:49:25.562009 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.561937 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62578128-8b67-4150-aeb4-12efb712bfe1-data-volume\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.562009 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:25.562011 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:26.061974171 +0000 UTC m=+252.857316897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : configmap "kube-rbac-proxy" not found Jul 24 13:49:25.562294 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.562278 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62578128-8b67-4150-aeb4-12efb712bfe1-data-volume\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.570630 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.570607 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-685w2\" (UniqueName: \"kubernetes.io/projected/62578128-8b67-4150-aeb4-12efb712bfe1-kube-api-access-685w2\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:25.636794 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.636754 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:25.736126 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:25.736095 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:26.065963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.065927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:26.066382 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.065997 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:26.066382 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:26.066062 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:49:26.066382 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:26.066088 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:49:26.066382 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:26.066113 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:27.066100262 +0000 UTC m=+253.861442970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : configmap "kube-rbac-proxy" not found Jul 24 13:49:26.066382 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:26.066145 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:27.066130767 +0000 UTC m=+253.861473475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : secret "insights-runtime-extractor-tls" not found Jul 24 13:49:26.104267 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.104230 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz"] Jul 24 13:49:26.106384 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.106369 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" Jul 24 13:49:26.108863 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.108839 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 13:49:26.110186 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.110166 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 13:49:26.110186 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.110173 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kgfm5\"" Jul 24 13:49:26.114888 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.114864 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz"] Jul 24 13:49:26.146544 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.146509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:26.166772 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.166732 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldf28\" (UniqueName: \"kubernetes.io/projected/c8c19755-f162-4a5b-a0f4-d503f7a270c5-kube-api-access-ldf28\") pod \"migrator-6865c9967-qzflz\" (UID: \"c8c19755-f162-4a5b-a0f4-d503f7a270c5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" Jul 24 13:49:26.248387 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.248343 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:26.267372 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.267343 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldf28\" (UniqueName: \"kubernetes.io/projected/c8c19755-f162-4a5b-a0f4-d503f7a270c5-kube-api-access-ldf28\") pod \"migrator-6865c9967-qzflz\" (UID: \"c8c19755-f162-4a5b-a0f4-d503f7a270c5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" Jul 24 13:49:26.277209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.277176 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldf28\" (UniqueName: \"kubernetes.io/projected/c8c19755-f162-4a5b-a0f4-d503f7a270c5-kube-api-access-ldf28\") pod \"migrator-6865c9967-qzflz\" (UID: \"c8c19755-f162-4a5b-a0f4-d503f7a270c5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" Jul 24 13:49:26.416869 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.416782 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" Jul 24 13:49:26.539795 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.539764 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz"] Jul 24 13:49:26.543590 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:49:26.543565 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c19755_f162_4a5b_a0f4_d503f7a270c5.slice/crio-c3fd2298b5e436761fb7ab6b60e21c2dce2a89eca50272d621173ad2269d26a9 WatchSource:0}: Error finding container c3fd2298b5e436761fb7ab6b60e21c2dce2a89eca50272d621173ad2269d26a9: Status 404 returned error can't find the container with id c3fd2298b5e436761fb7ab6b60e21c2dce2a89eca50272d621173ad2269d26a9 Jul 24 13:49:26.972371 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:26.972330 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:26.972559 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:26.972485 2555 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 24 13:49:26.972559 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:26.972552 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls podName:d4d4123c-fd9e-4f02-934c-5295f09a6bca nodeName:}" failed. No retries permitted until 2026-07-24 13:49:34.972535557 +0000 UTC m=+261.767878264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls") pod "cluster-samples-operator-6748489656-s6rhh" (UID: "d4d4123c-fd9e-4f02-934c-5295f09a6bca") : secret "samples-operator-tls" not found Jul 24 13:49:27.072830 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.072791 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:27.073276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.072863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:27.073276 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:27.072950 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:49:27.073276 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:27.073020 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:49:27.073426 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:27.073366 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:29.073287982 +0000 UTC m=+255.868630703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : configmap "kube-rbac-proxy" not found Jul 24 13:49:27.073426 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:27.073398 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:29.073377959 +0000 UTC m=+255.868720674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : secret "insights-runtime-extractor-tls" not found Jul 24 13:49:27.129851 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.129820 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cvnzc_01d4d8d2-be23-425c-b318-811a3cd71ab8/dns-node-resolver/0.log" Jul 24 13:49:27.222670 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.222591 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-crrjz"] Jul 24 13:49:27.224823 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.224800 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.227742 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.227649 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 13:49:27.227742 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.227649 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 13:49:27.227742 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.227652 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 13:49:27.227742 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.227652 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 13:49:27.228812 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.228791 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rrsrb\"" Jul 24 13:49:27.235575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.235552 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-crrjz"] Jul 24 13:49:27.256950 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.256907 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:27.315247 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.315214 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:27.374468 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.374425 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c291353-a542-4b27-86f6-292d470f3b80-signing-key\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.374648 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.374486 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c291353-a542-4b27-86f6-292d470f3b80-signing-cabundle\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.374648 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.374550 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5trk\" (UniqueName: \"kubernetes.io/projected/3c291353-a542-4b27-86f6-292d470f3b80-kube-api-access-x5trk\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.475680 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.475591 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c291353-a542-4b27-86f6-292d470f3b80-signing-key\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.475680 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.475648 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c291353-a542-4b27-86f6-292d470f3b80-signing-cabundle\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.475879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.475707 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5trk\" (UniqueName: \"kubernetes.io/projected/3c291353-a542-4b27-86f6-292d470f3b80-kube-api-access-x5trk\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.476499 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.476468 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c291353-a542-4b27-86f6-292d470f3b80-signing-cabundle\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.478738 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.478717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c291353-a542-4b27-86f6-292d470f3b80-signing-key\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.484263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.484240 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5trk\" (UniqueName: \"kubernetes.io/projected/3c291353-a542-4b27-86f6-292d470f3b80-kube-api-access-x5trk\") pod \"service-ca-658d7b7c78-crrjz\" (UID: \"3c291353-a542-4b27-86f6-292d470f3b80\") " pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.519254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.519219 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" event={"ID":"c8c19755-f162-4a5b-a0f4-d503f7a270c5","Type":"ContainerStarted","Data":"c3fd2298b5e436761fb7ab6b60e21c2dce2a89eca50272d621173ad2269d26a9"} Jul 24 13:49:27.536610 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.536574 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" Jul 24 13:49:27.856631 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:27.856603 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-crrjz"] Jul 24 13:49:27.857749 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:49:27.857723 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c291353_a542_4b27_86f6_292d470f3b80.slice/crio-d93870154c90cd51e5a09108c1c7995464a7f6c4bfdeff1a147dc24b66a72da3 WatchSource:0}: Error finding container d93870154c90cd51e5a09108c1c7995464a7f6c4bfdeff1a147dc24b66a72da3: Status 404 returned error can't find the container with id d93870154c90cd51e5a09108c1c7995464a7f6c4bfdeff1a147dc24b66a72da3 Jul 24 13:49:28.143678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:28.143649 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:28.523675 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:28.523580 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" event={"ID":"c8c19755-f162-4a5b-a0f4-d503f7a270c5","Type":"ContainerStarted","Data":"f6d00c9c1c85f7efc5ca5b8f36e1506a5e43d81a456510d0798051b914cbc1df"} Jul 24 13:49:28.523675 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:28.523640 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" event={"ID":"c8c19755-f162-4a5b-a0f4-d503f7a270c5","Type":"ContainerStarted","Data":"5812f72845934e5deb4d26f9d752a1f9e18fd044041892e5cff8e370405efdf6"} Jul 24 13:49:28.524644 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:28.524623 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" event={"ID":"3c291353-a542-4b27-86f6-292d470f3b80","Type":"ContainerStarted","Data":"d93870154c90cd51e5a09108c1c7995464a7f6c4bfdeff1a147dc24b66a72da3"} Jul 24 13:49:28.531076 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:28.531056 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n9vmm_be143ea6-d2ee-4709-8fd8-de79390abd1f/node-ca/0.log" Jul 24 13:49:28.542445 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:28.542385 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-qzflz" podStartSLOduration=1.354800348 podStartE2EDuration="2.542367965s" totalImagesPullingTime="1.187567617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:49:28.541164548 +0000 UTC m=+255.336507278" watchObservedRunningTime="2026-07-24 13:49:28.542367965 +0000 UTC m=+255.337710694" Jul 24 13:49:29.089241 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:29.089200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:29.089416 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:29.089264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:29.089416 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:29.089367 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:49:29.089416 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:29.089370 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:49:29.089524 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:29.089427 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:33.089410956 +0000 UTC m=+259.884753665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : configmap "kube-rbac-proxy" not found Jul 24 13:49:29.089524 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:29.089441 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:33.089435192 +0000 UTC m=+259.884777900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : secret "insights-runtime-extractor-tls" not found Jul 24 13:49:29.330406 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:29.330338 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xknnz_1c1a5e07-f00e-4480-b0f1-f2fa6624461a/serve-healthcheck-canary/0.log" Jul 24 13:49:30.532614 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:30.532573 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" event={"ID":"3c291353-a542-4b27-86f6-292d470f3b80","Type":"ContainerStarted","Data":"c374403e7efad06c0e6f10fb73591ea32d58ab49f7f0f98669d8f84897718bf2"} Jul 24 13:49:30.550710 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:30.550649 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-crrjz" podStartSLOduration=1.5956172259999999 podStartE2EDuration="3.550625254s" totalImagesPullingTime="1.955008028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:49:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:49:30.549053938 +0000 UTC m=+257.344396668" watchObservedRunningTime="2026-07-24 13:49:30.550625254 +0000 UTC m=+257.345967987" Jul 24 13:49:31.186045 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.185942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:31.336003 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.335940 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:31.574233 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.574202 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:31.667564 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.667531 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:31.766350 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.766307 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:31.894460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.894369 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:31.954228 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:31.954185 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:32.482458 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:32.482426 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:33.079663 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:33.079623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:33.126241 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:33.126206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:33.126412 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:33.126267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:33.126412 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:33.126336 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 13:49:33.126412 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:33.126381 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:49:33.126412 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:33.126401 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:41.12638523 +0000 UTC m=+267.921727938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : configmap "kube-rbac-proxy" not found Jul 24 13:49:33.126544 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:33.126441 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:41.126423672 +0000 UTC m=+267.921766386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : secret "insights-runtime-extractor-tls" not found Jul 24 13:49:33.681829 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:33.681796 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:33.882578 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:33.882503 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:34.078318 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:34.078285 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:34.481645 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:34.481562 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:35.043509 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.043472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:35.046307 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.046275 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d4d4123c-fd9e-4f02-934c-5295f09a6bca-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-s6rhh\" (UID: \"d4d4123c-fd9e-4f02-934c-5295f09a6bca\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:35.081882 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.081839 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" Jul 24 13:49:35.084348 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.084040 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:35.185243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.185211 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:35.214951 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.214920 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh"] Jul 24 13:49:35.482484 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.482400 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:35.550750 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.550716 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" event={"ID":"d4d4123c-fd9e-4f02-934c-5295f09a6bca","Type":"ContainerStarted","Data":"a3503b6adc8081b8c0e2f73165231f4606255fd03fc0334ee70cda98ed9e1a80"} Jul 24 13:49:35.880096 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:35.879968 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:36.087387 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:36.087340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:36.487796 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:36.487759 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:37.087864 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.087830 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:37.558071 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.558038 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" event={"ID":"d4d4123c-fd9e-4f02-934c-5295f09a6bca","Type":"ContainerStarted","Data":"45cba4cf012d359460d38bf30c435ceb5c6be12892c93dd388cf863be7abb479"} Jul 24 13:49:37.558452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.558089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" event={"ID":"d4d4123c-fd9e-4f02-934c-5295f09a6bca","Type":"ContainerStarted","Data":"a736c67b54ffe78f9073ef827579e328c3ca433ff4407b7d57fc7bac78e4abc1"} Jul 24 13:49:37.579717 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.577881 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-s6rhh" podStartSLOduration=16.918781261 podStartE2EDuration="18.577860715s" totalImagesPullingTime="1.659079454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:49:37.574972602 +0000 UTC m=+264.370315332" watchObservedRunningTime="2026-07-24 13:49:37.577860715 +0000 UTC m=+264.373203446" Jul 24 13:49:37.677262 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.677203 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:37.728373 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.728338 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:37.886005 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:37.885904 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:38.082153 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:38.082116 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:38.201297 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:38.201216 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:38.491202 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:38.491117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:39.083385 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:39.083352 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:39.495503 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:39.495425 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:39.829345 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:39.829306 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:39.931203 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:39.931170 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:40.280044 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:40.279935 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:40.483275 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:40.483248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:40.890294 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:40.890263 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:41.192692 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.192599 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:41.192692 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.192656 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:41.192891 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:41.192758 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 13:49:41.192891 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:41.192836 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls podName:62578128-8b67-4150-aeb4-12efb712bfe1 nodeName:}" failed. No retries permitted until 2026-07-24 13:49:57.192820655 +0000 UTC m=+283.988163362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqc9b" (UID: "62578128-8b67-4150-aeb4-12efb712bfe1") : secret "insights-runtime-extractor-tls" not found Jul 24 13:49:41.193177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.193159 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62578128-8b67-4150-aeb4-12efb712bfe1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:41.294968 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.294930 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:41.632884 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.632855 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:41.733688 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.733655 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:41.881740 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:41.881706 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:43.414505 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:43.414475 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:47.212202 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.212166 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6564b854b7-j9x9w"] Jul 24 13:49:47.214341 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.214325 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.217817 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.217792 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 13:49:47.217946 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.217823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 13:49:47.218513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.218498 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bz29t\"" Jul 24 13:49:47.218622 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.218498 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 13:49:47.225554 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.225528 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 13:49:47.230200 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.230173 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6564b854b7-j9x9w"] Jul 24 13:49:47.274247 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.274204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:47.343050 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343014 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtmzp\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-kube-api-access-vtmzp\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343050 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343052 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-registry-tls\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343080 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-registry-certificates\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343120 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-installation-pull-secrets\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343143 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-trusted-ca\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-image-registry-private-configuration\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343233 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-ca-trust-extracted\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.343263 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.343262 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-bound-sa-token\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.399295 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.399258 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:47.444091 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtmzp\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-kube-api-access-vtmzp\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-registry-tls\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444146 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-registry-certificates\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444198 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-installation-pull-secrets\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444231 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-trusted-ca\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-image-registry-private-configuration\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444300 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444293 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-ca-trust-extracted\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.444584 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.444319 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-bound-sa-token\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.445067 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.445038 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-registry-certificates\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.445330 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.445296 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-ca-trust-extracted\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.445622 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.445592 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-trusted-ca\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.447038 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.447011 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-image-registry-private-configuration\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.447149 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.447040 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-registry-tls\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.447149 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.447044 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-installation-pull-secrets\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.455251 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.455224 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-bound-sa-token\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.455465 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.455353 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtmzp\" (UniqueName: \"kubernetes.io/projected/9ff6530b-8347-4bbf-a3ec-934e0a1b898d-kube-api-access-vtmzp\") pod \"image-registry-6564b854b7-j9x9w\" (UID: \"9ff6530b-8347-4bbf-a3ec-934e0a1b898d\") " pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.523406 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.523323 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:47.658081 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:47.658043 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6564b854b7-j9x9w"] Jul 24 13:49:47.661288 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:49:47.661264 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff6530b_8347_4bbf_a3ec_934e0a1b898d.slice/crio-001685e7411abd8057419b742c8e0c317ae79cc8842b8b55cf5a18247e337bdd WatchSource:0}: Error finding container 001685e7411abd8057419b742c8e0c317ae79cc8842b8b55cf5a18247e337bdd: Status 404 returned error can't find the container with id 001685e7411abd8057419b742c8e0c317ae79cc8842b8b55cf5a18247e337bdd Jul 24 13:49:48.594899 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:48.594864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" event={"ID":"9ff6530b-8347-4bbf-a3ec-934e0a1b898d","Type":"ContainerStarted","Data":"a33e13827cb46f3291ba719e3c32c20a0795f3709ac9898a5dd8c656f8d3d446"} Jul 24 13:49:48.595286 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:48.594913 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" event={"ID":"9ff6530b-8347-4bbf-a3ec-934e0a1b898d","Type":"ContainerStarted","Data":"001685e7411abd8057419b742c8e0c317ae79cc8842b8b55cf5a18247e337bdd"} Jul 24 13:49:48.595286 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:48.595024 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:49:48.613933 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:48.613885 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" podStartSLOduration=1.613868859 podStartE2EDuration="1.613868859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:49:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:49:48.613007709 +0000 UTC m=+275.408350434" watchObservedRunningTime="2026-07-24 13:49:48.613868859 +0000 UTC m=+275.409211589" Jul 24 13:49:53.532700 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:53.532660 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:53.632372 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:53.632342 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:53.699818 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:53.699787 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:53.828343 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:53.828315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:54.252781 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:49:54.252672 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-sc55j" podUID="4949ccfd-f2b4-44b4-9e00-2228bb2ad367" Jul 24 13:49:54.614401 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:54.614372 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sc55j" Jul 24 13:49:56.728186 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:56.728134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:56.807880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:56.807838 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:56.939305 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:56.939272 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:49:57.222306 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:57.222271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:57.224789 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:57.224756 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62578128-8b67-4150-aeb4-12efb712bfe1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqc9b\" (UID: \"62578128-8b67-4150-aeb4-12efb712bfe1\") " pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:57.458819 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:57.458758 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqc9b" Jul 24 13:49:57.593251 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:57.593219 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqc9b"] Jul 24 13:49:57.596566 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:49:57.596538 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62578128_8b67_4150_aeb4_12efb712bfe1.slice/crio-fd51f5c5704d7163c26ced53d436d6be9f26e582b8c615f24821e8de51ff2c0f WatchSource:0}: Error finding container fd51f5c5704d7163c26ced53d436d6be9f26e582b8c615f24821e8de51ff2c0f: Status 404 returned error can't find the container with id fd51f5c5704d7163c26ced53d436d6be9f26e582b8c615f24821e8de51ff2c0f Jul 24 13:49:57.625775 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:57.625742 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqc9b" event={"ID":"62578128-8b67-4150-aeb4-12efb712bfe1","Type":"ContainerStarted","Data":"fd51f5c5704d7163c26ced53d436d6be9f26e582b8c615f24821e8de51ff2c0f"} Jul 24 13:49:58.129099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.129058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:49:58.132495 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.132464 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4949ccfd-f2b4-44b4-9e00-2228bb2ad367-metrics-tls\") pod \"dns-default-sc55j\" (UID: \"4949ccfd-f2b4-44b4-9e00-2228bb2ad367\") " pod="openshift-dns/dns-default-sc55j" Jul 24 13:49:58.218148 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.218113 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7f5lp\"" Jul 24 13:49:58.225584 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.225550 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sc55j" Jul 24 13:49:58.404883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.404760 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sc55j"] Jul 24 13:49:58.409769 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:49:58.409732 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4949ccfd_f2b4_44b4_9e00_2228bb2ad367.slice/crio-0ba04a3558e09838a67a8fcc10815df6a66e9ad0803a086713f5257dfae296b2 WatchSource:0}: Error finding container 0ba04a3558e09838a67a8fcc10815df6a66e9ad0803a086713f5257dfae296b2: Status 404 returned error can't find the container with id 0ba04a3558e09838a67a8fcc10815df6a66e9ad0803a086713f5257dfae296b2 Jul 24 13:49:58.632033 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.631969 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sc55j" event={"ID":"4949ccfd-f2b4-44b4-9e00-2228bb2ad367","Type":"ContainerStarted","Data":"0ba04a3558e09838a67a8fcc10815df6a66e9ad0803a086713f5257dfae296b2"} Jul 24 13:49:58.953798 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:49:58.953743 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:00.935963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:00.935914 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:01.075143 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.075108 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:01.170575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.170540 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:01.210765 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.208324 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-85q2q"] Jul 24 13:50:01.229786 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.229747 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.235351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.234783 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 13:50:01.235351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.235043 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 13:50:01.235351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.235206 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dgt7z\"" Jul 24 13:50:01.235351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.235241 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 13:50:01.236668 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.236492 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 13:50:01.236796 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.236763 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 13:50:01.236963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.236825 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-wtmp\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258177 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-accelerators-collector-config\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258214 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-root\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258266 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06071628-5820-4d75-9ff3-c1519047d07a-metrics-client-ca\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258303 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-tls\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258331 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-sys\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-textfile\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.258513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.258401 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnb9j\" (UniqueName: \"kubernetes.io/projected/06071628-5820-4d75-9ff3-c1519047d07a-kube-api-access-hnb9j\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359684 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359645 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06071628-5820-4d75-9ff3-c1519047d07a-metrics-client-ca\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-tls\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359723 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-sys\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-textfile\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnb9j\" (UniqueName: \"kubernetes.io/projected/06071628-5820-4d75-9ff3-c1519047d07a-kube-api-access-hnb9j\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359836 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-wtmp\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:50:01.359866 2555 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 24 13:50:01.359880 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359872 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-accelerators-collector-config\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359907 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:50:01.359930 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-tls podName:06071628-5820-4d75-9ff3-c1519047d07a nodeName:}" failed. No retries permitted until 2026-07-24 13:50:01.859911502 +0000 UTC m=+288.655254224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-tls") pod "node-exporter-85q2q" (UID: "06071628-5820-4d75-9ff3-c1519047d07a") : secret "node-exporter-tls" not found Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.359967 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-root\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.360080 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-root\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.360279 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-sys\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.360416 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-wtmp\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.360418 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06071628-5820-4d75-9ff3-c1519047d07a-metrics-client-ca\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.360516 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-textfile\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.361243 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.360784 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-accelerators-collector-config\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.362503 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.362472 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.371751 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.371720 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnb9j\" (UniqueName: \"kubernetes.io/projected/06071628-5820-4d75-9ff3-c1519047d07a-kube-api-access-hnb9j\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.553834 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.553735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:01.688339 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.688304 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:01.747061 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.747029 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:01.865248 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.865154 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-tls\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.867934 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.867900 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/06071628-5820-4d75-9ff3-c1519047d07a-node-exporter-tls\") pod \"node-exporter-85q2q\" (UID: \"06071628-5820-4d75-9ff3-c1519047d07a\") " pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:01.947635 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:01.947599 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:02.144130 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:02.144037 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-85q2q" Jul 24 13:50:02.298894 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:02.298857 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:02.400214 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:02.400130 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:02.876376 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:50:02.876345 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06071628_5820_4d75_9ff3_c1519047d07a.slice/crio-0d0c63fecabf32f39bb2636a7240ad0afceb96134f9a1bc495ba3e2adafe5cc1 WatchSource:0}: Error finding container 0d0c63fecabf32f39bb2636a7240ad0afceb96134f9a1bc495ba3e2adafe5cc1: Status 404 returned error can't find the container with id 0d0c63fecabf32f39bb2636a7240ad0afceb96134f9a1bc495ba3e2adafe5cc1 Jul 24 13:50:03.655585 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:03.655479 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqc9b" event={"ID":"62578128-8b67-4150-aeb4-12efb712bfe1","Type":"ContainerStarted","Data":"31af4df4faa5630cc81755c9fd3561f0eeae63ff829fb745b24a538bddf0e1fa"} Jul 24 13:50:03.659247 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:03.658349 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sc55j" event={"ID":"4949ccfd-f2b4-44b4-9e00-2228bb2ad367","Type":"ContainerStarted","Data":"4d11dd64f30577084b52e46868707e4bd3c4ccb82efb1def9d4abea9c3b757ec"} Jul 24 13:50:03.659247 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:03.658408 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sc55j" event={"ID":"4949ccfd-f2b4-44b4-9e00-2228bb2ad367","Type":"ContainerStarted","Data":"35f00a5f98866deb0c27d1a9dad94b94e7f69e6c2c23e3468bc2a5ed9a18ff66"} Jul 24 13:50:03.659247 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:03.659182 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sc55j" Jul 24 13:50:03.661766 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:03.661737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-85q2q" event={"ID":"06071628-5820-4d75-9ff3-c1519047d07a","Type":"ContainerStarted","Data":"0d0c63fecabf32f39bb2636a7240ad0afceb96134f9a1bc495ba3e2adafe5cc1"} Jul 24 13:50:04.116445 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.116413 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:04.216909 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.216870 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:04.414161 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.414128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:04.668175 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.668081 2555 generic.go:356] "Generic (PLEG): container finished" podID="06071628-5820-4d75-9ff3-c1519047d07a" containerID="bbfc72f89465916bee392b941ab425d8c662ff00e2260b84388441dffb533e3f" exitCode=0 Jul 24 13:50:04.668623 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.668171 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-85q2q" event={"ID":"06071628-5820-4d75-9ff3-c1519047d07a","Type":"ContainerDied","Data":"bbfc72f89465916bee392b941ab425d8c662ff00e2260b84388441dffb533e3f"} Jul 24 13:50:04.688532 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.688478 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sc55j" podStartSLOduration=252.229664937 podStartE2EDuration="4m16.688462562s" totalImagesPullingTime="4.458797625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:45:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:03.67758859 +0000 UTC m=+290.472931322" watchObservedRunningTime="2026-07-24 13:50:04.688462562 +0000 UTC m=+291.483805293" Jul 24 13:50:04.931879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:04.931799 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:05.558938 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.558909 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7788c867cc-6sqck"] Jul 24 13:50:05.577188 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.577151 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7788c867cc-6sqck"] Jul 24 13:50:05.577360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.577295 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.581460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.581319 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 24 13:50:05.581636 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.581558 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jc4p8\"" Jul 24 13:50:05.581751 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.581675 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-d46eu251845gr\"" Jul 24 13:50:05.581751 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.581689 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 24 13:50:05.581862 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.581773 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 24 13:50:05.582049 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.582022 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 24 13:50:05.673967 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.673925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-85q2q" event={"ID":"06071628-5820-4d75-9ff3-c1519047d07a","Type":"ContainerStarted","Data":"dac79979082e6cac3503873304998a8f9a263ad0c327f0f94fc297fd10fa6ec8"} Jul 24 13:50:05.698280 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698238 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-client-ca-bundle\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.698467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698300 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-secret-metrics-server-client-certs\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.698467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698335 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8564a70-37c5-45a1-bb1f-98f092faef66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.698467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8564a70-37c5-45a1-bb1f-98f092faef66-audit-log\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.698467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698385 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-secret-metrics-server-tls\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.698696 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8564a70-37c5-45a1-bb1f-98f092faef66-metrics-server-audit-profiles\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.698696 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.698553 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7cm6\" (UniqueName: \"kubernetes.io/projected/c8564a70-37c5-45a1-bb1f-98f092faef66-kube-api-access-l7cm6\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799105 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799065 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-client-ca-bundle\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799119 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-secret-metrics-server-client-certs\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799156 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8564a70-37c5-45a1-bb1f-98f092faef66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8564a70-37c5-45a1-bb1f-98f092faef66-audit-log\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-secret-metrics-server-tls\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799282 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8564a70-37c5-45a1-bb1f-98f092faef66-metrics-server-audit-profiles\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.799557 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799314 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7cm6\" (UniqueName: \"kubernetes.io/projected/c8564a70-37c5-45a1-bb1f-98f092faef66-kube-api-access-l7cm6\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.800014 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.799669 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c8564a70-37c5-45a1-bb1f-98f092faef66-audit-log\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.800143 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.800029 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8564a70-37c5-45a1-bb1f-98f092faef66-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.800802 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.800773 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c8564a70-37c5-45a1-bb1f-98f092faef66-metrics-server-audit-profiles\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.802192 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.802162 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-client-ca-bundle\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.802487 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.802464 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-secret-metrics-server-client-certs\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.802841 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.802823 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c8564a70-37c5-45a1-bb1f-98f092faef66-secret-metrics-server-tls\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.808475 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.808445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7cm6\" (UniqueName: \"kubernetes.io/projected/c8564a70-37c5-45a1-bb1f-98f092faef66-kube-api-access-l7cm6\") pod \"metrics-server-7788c867cc-6sqck\" (UID: \"c8564a70-37c5-45a1-bb1f-98f092faef66\") " pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.889010 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.888919 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:05.921971 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:05.921934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:06.550084 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:06.550041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:06.745962 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:50:06.745904 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ql85p" podUID="3c5e9d55-41ee-4f12-ae25-74947f9bc6f5" Jul 24 13:50:06.849523 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:06.849500 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7788c867cc-6sqck"] Jul 24 13:50:06.852191 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:50:06.852160 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8564a70_37c5_45a1_bb1f_98f092faef66.slice/crio-fa54f4309fafd3f42178e32d13e7724d832e2d5054d00ba030a99c941e9bb4e2 WatchSource:0}: Error finding container fa54f4309fafd3f42178e32d13e7724d832e2d5054d00ba030a99c941e9bb4e2: Status 404 returned error can't find the container with id fa54f4309fafd3f42178e32d13e7724d832e2d5054d00ba030a99c941e9bb4e2 Jul 24 13:50:07.367906 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.367872 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:07.530614 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.530558 2555 patch_prober.go:28] interesting pod/image-registry-6564b854b7-j9x9w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:50:07.530807 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.530628 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" podUID="9ff6530b-8347-4bbf-a3ec-934e0a1b898d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:50:07.685726 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.685635 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqc9b" event={"ID":"62578128-8b67-4150-aeb4-12efb712bfe1","Type":"ContainerStarted","Data":"2de1023409350c59cfe8c5faa7544d315d5d94300541e0c5359176276b40788b"} Jul 24 13:50:07.689524 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.689454 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-85q2q" event={"ID":"06071628-5820-4d75-9ff3-c1519047d07a","Type":"ContainerStarted","Data":"b8c7aa33abd6271a7bf1ba55aa8b2629166409e1bacdab88a558d1aebf73e06f"} Jul 24 13:50:07.692236 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.692198 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" event={"ID":"c8564a70-37c5-45a1-bb1f-98f092faef66","Type":"ContainerStarted","Data":"fa54f4309fafd3f42178e32d13e7724d832e2d5054d00ba030a99c941e9bb4e2"} Jul 24 13:50:07.710842 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:07.710770 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-85q2q" podStartSLOduration=5.510521251 podStartE2EDuration="6.710745367s" totalImagesPullingTime="1.091826735s" totalInitContainerRuntime="108.397381ms" isStatefulPod=true podCreationTimestamp="2026-07-24 13:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:07.708571032 +0000 UTC m=+294.503913755" watchObservedRunningTime="2026-07-24 13:50:07.710745367 +0000 UTC m=+294.506088098" Jul 24 13:50:08.699605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:08.699563 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" event={"ID":"c8564a70-37c5-45a1-bb1f-98f092faef66","Type":"ContainerStarted","Data":"da679e8f29ee7d393f1da1f9c70db5677dac55fb1c941dd245132d62a094e860"} Jul 24 13:50:08.724849 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:08.724782 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" podStartSLOduration=2.28560292 podStartE2EDuration="3.724758054s" totalImagesPullingTime="1.439155134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:50:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:08.717194432 +0000 UTC m=+295.512537164" watchObservedRunningTime="2026-07-24 13:50:08.724758054 +0000 UTC m=+295.520100785" Jul 24 13:50:09.603600 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:09.603562 2555 patch_prober.go:28] interesting pod/image-registry-6564b854b7-j9x9w container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:50:09.603773 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:09.603630 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" podUID="9ff6530b-8347-4bbf-a3ec-934e0a1b898d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:50:10.709575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:10.709544 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqc9b" event={"ID":"62578128-8b67-4150-aeb4-12efb712bfe1","Type":"ContainerStarted","Data":"a8e0a826896f923bf9e4182111712dc087481bc96cb31c090b10b07edb15ce04"} Jul 24 13:50:11.631537 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:11.631501 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:11.690340 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:11.690305 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:11.733031 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:11.732957 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bqc9b" podStartSLOduration=33.830913816 podStartE2EDuration="46.732940201s" totalImagesPullingTime="12.902026385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:49:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:11.731875167 +0000 UTC m=+298.527217896" watchObservedRunningTime="2026-07-24 13:50:11.732940201 +0000 UTC m=+298.528282932" Jul 24 13:50:13.584299 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:13.584270 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 13:50:14.675873 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:14.675849 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sc55j" Jul 24 13:50:17.179044 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.179005 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:17.240209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.240174 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:17.308132 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.308098 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:17.449444 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.449354 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:17.528415 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.528380 2555 patch_prober.go:28] interesting pod/image-registry-6564b854b7-j9x9w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:50:17.528583 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.528441 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" podUID="9ff6530b-8347-4bbf-a3ec-934e0a1b898d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:50:17.626156 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.626125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:17.687460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:17.687421 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:18.022839 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:18.022805 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:18.083377 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:18.083348 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:19.094621 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.094586 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:19.154436 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.154400 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:19.232042 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.232009 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:19.487962 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.487871 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:19.602587 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.602560 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6564b854b7-j9x9w" Jul 24 13:50:19.738907 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.738813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:19.838414 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:19.838381 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:20.362810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:20.362775 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:20.507507 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:20.507466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:21.446092 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:21.446061 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:21.743210 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:21.743129 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:50:23.116004 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:23.115934 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:23.217184 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:23.217148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:23.834122 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:23.834086 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:23.935378 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:23.935345 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:25.889144 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:25.889111 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:25.889598 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:25.889174 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:26.872792 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.872754 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5db46b59d4-l9nwn"] Jul 24 13:50:26.875870 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.875850 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:26.878592 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.878569 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 13:50:26.879808 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.879783 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 13:50:26.879928 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.879793 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 13:50:26.879928 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.879874 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 13:50:26.879928 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.879915 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 13:50:26.880113 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.879932 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nnrjg\"" Jul 24 13:50:26.880454 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.880440 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 13:50:26.880500 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.880472 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 13:50:26.886477 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.886450 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db46b59d4-l9nwn"] Jul 24 13:50:26.991910 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.991869 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-config\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:26.991910 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.991905 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd9wv\" (UniqueName: \"kubernetes.io/projected/dc49be88-2253-4981-81a7-403e0d2dbc7c-kube-api-access-xd9wv\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:26.992360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.991929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-oauth-serving-cert\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:26.992360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.992021 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-service-ca\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:26.992360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.992089 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-serving-cert\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:26.992360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:26.992164 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-oauth-config\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.093607 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.093571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-serving-cert\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.093809 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.093654 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-oauth-config\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.093809 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.093701 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-config\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.093809 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.093726 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd9wv\" (UniqueName: \"kubernetes.io/projected/dc49be88-2253-4981-81a7-403e0d2dbc7c-kube-api-access-xd9wv\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.093809 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.093794 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-oauth-serving-cert\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.094180 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.093844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-service-ca\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.094587 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.094561 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-config\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.094749 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.094664 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-oauth-serving-cert\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.094749 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.094742 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-service-ca\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.096486 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.096459 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-oauth-config\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.096622 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.096599 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-serving-cert\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.102748 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.102717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd9wv\" (UniqueName: \"kubernetes.io/projected/dc49be88-2253-4981-81a7-403e0d2dbc7c-kube-api-access-xd9wv\") pod \"console-5db46b59d4-l9nwn\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.186218 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.186123 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:27.327807 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.327759 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5db46b59d4-l9nwn"] Jul 24 13:50:27.330567 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:50:27.330539 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc49be88_2253_4981_81a7_403e0d2dbc7c.slice/crio-3dce0968588147b4b24e4222ddd4b651b76c2e537e71cb835b8c48dea16c631e WatchSource:0}: Error finding container 3dce0968588147b4b24e4222ddd4b651b76c2e537e71cb835b8c48dea16c631e: Status 404 returned error can't find the container with id 3dce0968588147b4b24e4222ddd4b651b76c2e537e71cb835b8c48dea16c631e Jul 24 13:50:27.332773 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.332754 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:50:27.768764 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:27.768730 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db46b59d4-l9nwn" event={"ID":"dc49be88-2253-4981-81a7-403e0d2dbc7c","Type":"ContainerStarted","Data":"3dce0968588147b4b24e4222ddd4b651b76c2e537e71cb835b8c48dea16c631e"} Jul 24 13:50:29.425623 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:29.425592 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:29.526569 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:29.526493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:30.659288 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:30.659249 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:30.758479 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:30.758450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:30.784227 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:30.784192 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db46b59d4-l9nwn" event={"ID":"dc49be88-2253-4981-81a7-403e0d2dbc7c","Type":"ContainerStarted","Data":"9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01"} Jul 24 13:50:30.801860 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:30.801803 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5db46b59d4-l9nwn" podStartSLOduration=1.8897993720000001 podStartE2EDuration="4.801783731s" totalImagesPullingTime="2.911984359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:30.800815564 +0000 UTC m=+317.596158296" watchObservedRunningTime="2026-07-24 13:50:30.801783731 +0000 UTC m=+317.597126460" Jul 24 13:50:32.834424 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:32.834386 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:32.935066 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:32.935035 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:34.250933 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.250895 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:34.303469 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.303433 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:34.657028 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.656996 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db59c56b8-ww9wd"] Jul 24 13:50:34.726921 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.726879 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db59c56b8-ww9wd"] Jul 24 13:50:34.727077 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.727001 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.734975 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.734953 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 13:50:34.761523 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-oauth-config\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.761691 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761529 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whmkx\" (UniqueName: \"kubernetes.io/projected/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-kube-api-access-whmkx\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.761691 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761588 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-serving-cert\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.761691 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-config\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.761691 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761659 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-oauth-serving-cert\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.761822 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761698 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-trusted-ca-bundle\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.761822 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.761719 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-service-ca\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862085 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-serving-cert\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862278 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862105 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-config\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862278 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862142 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-oauth-serving-cert\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862278 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862189 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-trusted-ca-bundle\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862278 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862211 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-service-ca\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862278 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-oauth-config\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.862507 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.862305 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whmkx\" (UniqueName: \"kubernetes.io/projected/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-kube-api-access-whmkx\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.863063 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.863034 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-service-ca\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.863254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.863236 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-trusted-ca-bundle\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.863304 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.863280 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-oauth-serving-cert\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.863447 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.863426 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-config\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.864815 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.864792 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-serving-cert\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.864897 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.864813 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-oauth-config\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:34.871113 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:34.871086 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whmkx\" (UniqueName: \"kubernetes.io/projected/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-kube-api-access-whmkx\") pod \"console-6db59c56b8-ww9wd\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:35.036008 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.035895 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:35.189064 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.189037 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db59c56b8-ww9wd"] Jul 24 13:50:35.191258 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:50:35.191235 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a7fb42_6fd7_4336_94f9_34c7f84cdd70.slice/crio-751ada6331565c345fd6e285b7229c019d652f0cab18c9b0154cc8844b1eabf3 WatchSource:0}: Error finding container 751ada6331565c345fd6e285b7229c019d652f0cab18c9b0154cc8844b1eabf3: Status 404 returned error can't find the container with id 751ada6331565c345fd6e285b7229c019d652f0cab18c9b0154cc8844b1eabf3 Jul 24 13:50:35.471594 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.471558 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xknnz_1c1a5e07-f00e-4480-b0f1-f2fa6624461a/serve-healthcheck-canary/0.log" Jul 24 13:50:35.803513 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.803484 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db59c56b8-ww9wd" event={"ID":"00a7fb42-6fd7-4336-94f9-34c7f84cdd70","Type":"ContainerStarted","Data":"ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0"} Jul 24 13:50:35.803685 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.803529 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db59c56b8-ww9wd" event={"ID":"00a7fb42-6fd7-4336-94f9-34c7f84cdd70","Type":"ContainerStarted","Data":"751ada6331565c345fd6e285b7229c019d652f0cab18c9b0154cc8844b1eabf3"} Jul 24 13:50:35.826447 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.826402 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6db59c56b8-ww9wd" podStartSLOduration=1.826387763 podStartE2EDuration="1.826387763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:50:35.824525995 +0000 UTC m=+322.619868730" watchObservedRunningTime="2026-07-24 13:50:35.826387763 +0000 UTC m=+322.621730493" Jul 24 13:50:35.863672 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:35.863641 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:37.186357 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:37.186313 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:37.186816 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:37.186394 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:37.191011 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:37.190975 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:37.813582 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:37.813558 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:50:41.266121 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:41.266083 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:41.324638 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:41.324607 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:41.487319 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:41.487279 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:41.547641 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:41.547599 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:41.637917 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:41.637886 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:41.738626 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:41.738591 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:45.036233 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.036189 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:45.036627 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.036299 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:45.040966 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.040944 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:45.842089 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.842058 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:50:45.893963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.893929 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db46b59d4-l9nwn"] Jul 24 13:50:45.895430 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.895393 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:45.900568 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:45.900545 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7788c867cc-6sqck" Jul 24 13:50:46.010107 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:46.010074 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:46.111151 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:46.111072 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:46.216310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:46.216272 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:56.652326 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:56.652286 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:50:56.750692 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:50:56.750652 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:04.202539 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:04.202454 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:04.304557 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:04.304523 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:04.406140 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:04.406103 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:07.617703 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:07.617651 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:07.716177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:07.716152 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:10.916586 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:10.916536 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5db46b59d4-l9nwn" podUID="dc49be88-2253-4981-81a7-403e0d2dbc7c" containerName="console" containerID="cri-o://9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01" gracePeriod=15 Jul 24 13:51:11.157036 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.157012 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:51:11.266185 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266103 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-oauth-config\") pod \"dc49be88-2253-4981-81a7-403e0d2dbc7c\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " Jul 24 13:51:11.266185 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266150 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-oauth-serving-cert\") pod \"dc49be88-2253-4981-81a7-403e0d2dbc7c\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " Jul 24 13:51:11.266185 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266169 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-serving-cert\") pod \"dc49be88-2253-4981-81a7-403e0d2dbc7c\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " Jul 24 13:51:11.266404 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266313 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-service-ca\") pod \"dc49be88-2253-4981-81a7-403e0d2dbc7c\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " Jul 24 13:51:11.266404 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266374 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-config\") pod \"dc49be88-2253-4981-81a7-403e0d2dbc7c\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " Jul 24 13:51:11.266504 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266403 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd9wv\" (UniqueName: \"kubernetes.io/projected/dc49be88-2253-4981-81a7-403e0d2dbc7c-kube-api-access-xd9wv\") pod \"dc49be88-2253-4981-81a7-403e0d2dbc7c\" (UID: \"dc49be88-2253-4981-81a7-403e0d2dbc7c\") " Jul 24 13:51:11.266657 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266609 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "dc49be88-2253-4981-81a7-403e0d2dbc7c" (UID: "dc49be88-2253-4981-81a7-403e0d2dbc7c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:51:11.266782 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266730 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-service-ca" (OuterVolumeSpecName: "service-ca") pod "dc49be88-2253-4981-81a7-403e0d2dbc7c" (UID: "dc49be88-2253-4981-81a7-403e0d2dbc7c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:51:11.266835 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.266770 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-config" (OuterVolumeSpecName: "console-config") pod "dc49be88-2253-4981-81a7-403e0d2dbc7c" (UID: "dc49be88-2253-4981-81a7-403e0d2dbc7c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:51:11.268639 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.268614 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "dc49be88-2253-4981-81a7-403e0d2dbc7c" (UID: "dc49be88-2253-4981-81a7-403e0d2dbc7c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:51:11.268639 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.268630 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "dc49be88-2253-4981-81a7-403e0d2dbc7c" (UID: "dc49be88-2253-4981-81a7-403e0d2dbc7c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:51:11.268762 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.268685 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc49be88-2253-4981-81a7-403e0d2dbc7c-kube-api-access-xd9wv" (OuterVolumeSpecName: "kube-api-access-xd9wv") pod "dc49be88-2253-4981-81a7-403e0d2dbc7c" (UID: "dc49be88-2253-4981-81a7-403e0d2dbc7c"). InnerVolumeSpecName "kube-api-access-xd9wv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:51:11.367782 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.367742 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-oauth-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:51:11.367782 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.367773 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-oauth-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:51:11.367782 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.367783 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:51:11.367782 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.367792 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-service-ca\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:51:11.368094 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.367801 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/dc49be88-2253-4981-81a7-403e0d2dbc7c-console-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:51:11.368094 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.367809 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd9wv\" (UniqueName: \"kubernetes.io/projected/dc49be88-2253-4981-81a7-403e0d2dbc7c-kube-api-access-xd9wv\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:51:11.929900 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.929865 2555 generic.go:356] "Generic (PLEG): container finished" podID="dc49be88-2253-4981-81a7-403e0d2dbc7c" containerID="9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01" exitCode=0 Jul 24 13:51:11.930359 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.929907 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5db46b59d4-l9nwn" Jul 24 13:51:11.930359 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.929944 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db46b59d4-l9nwn" event={"ID":"dc49be88-2253-4981-81a7-403e0d2dbc7c","Type":"ContainerDied","Data":"9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01"} Jul 24 13:51:11.930359 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.930014 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5db46b59d4-l9nwn" event={"ID":"dc49be88-2253-4981-81a7-403e0d2dbc7c","Type":"ContainerDied","Data":"3dce0968588147b4b24e4222ddd4b651b76c2e537e71cb835b8c48dea16c631e"} Jul 24 13:51:11.930359 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.930047 2555 scope.go:117] "RemoveContainer" containerID="9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01" Jul 24 13:51:11.938248 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.938229 2555 scope.go:117] "RemoveContainer" containerID="9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01" Jul 24 13:51:11.938559 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:51:11.938535 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01\": container with ID starting with 9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01 not found: ID does not exist" containerID="9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01" Jul 24 13:51:11.938666 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.938567 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01"} err="failed to get container status \"9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01\": rpc error: code = NotFound desc = could not find container \"9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01\": container with ID starting with 9c976fd4210688a4595dec10f63142d693a115f5b753015450320d7f8708ba01 not found: ID does not exist" Jul 24 13:51:11.946758 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.946730 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5db46b59d4-l9nwn"] Jul 24 13:51:11.950374 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:11.950349 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5db46b59d4-l9nwn"] Jul 24 13:51:13.748597 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:13.748560 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc49be88-2253-4981-81a7-403e0d2dbc7c" path="/var/lib/kubelet/pods/dc49be88-2253-4981-81a7-403e0d2dbc7c/volumes" Jul 24 13:51:20.397118 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:20.397076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:20.671659 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:20.671583 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:27.508754 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:27.508712 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:51:27.511125 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:27.511105 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c5e9d55-41ee-4f12-ae25-74947f9bc6f5-metrics-certs\") pod \"network-metrics-daemon-ql85p\" (UID: \"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5\") " pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:51:27.746921 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:27.746882 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qbf75\"" Jul 24 13:51:27.754361 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:27.754341 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ql85p" Jul 24 13:51:27.908304 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:27.908280 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ql85p"] Jul 24 13:51:27.913269 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:51:27.913237 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5e9d55_41ee_4f12_ae25_74947f9bc6f5.slice/crio-dd5602b733265a2b8aab989656989e033ae5b43a18eaaa956ff3ea098450f444 WatchSource:0}: Error finding container dd5602b733265a2b8aab989656989e033ae5b43a18eaaa956ff3ea098450f444: Status 404 returned error can't find the container with id dd5602b733265a2b8aab989656989e033ae5b43a18eaaa956ff3ea098450f444 Jul 24 13:51:27.988629 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:27.988592 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ql85p" event={"ID":"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5","Type":"ContainerStarted","Data":"dd5602b733265a2b8aab989656989e033ae5b43a18eaaa956ff3ea098450f444"} Jul 24 13:51:29.172399 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.172366 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86db4db4fb-xfwbw"] Jul 24 13:51:29.172879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.172789 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc49be88-2253-4981-81a7-403e0d2dbc7c" containerName="console" Jul 24 13:51:29.172879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.172810 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc49be88-2253-4981-81a7-403e0d2dbc7c" containerName="console" Jul 24 13:51:29.172879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.172872 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc49be88-2253-4981-81a7-403e0d2dbc7c" containerName="console" Jul 24 13:51:29.174904 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.174881 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.194252 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.194225 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86db4db4fb-xfwbw"] Jul 24 13:51:29.222273 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222235 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-oauth-config\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.222431 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-oauth-serving-cert\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.222431 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222306 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-serving-cert\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.222431 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222356 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2jtr\" (UniqueName: \"kubernetes.io/projected/af09956d-eb6f-4497-b917-e9f20df02c77-kube-api-access-x2jtr\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.222431 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222371 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-console-config\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.222431 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222388 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-trusted-ca-bundle\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.222627 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.222438 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-service-ca\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.239931 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.239900 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:29.323708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2jtr\" (UniqueName: \"kubernetes.io/projected/af09956d-eb6f-4497-b917-e9f20df02c77-kube-api-access-x2jtr\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.323708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323707 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-console-config\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.323958 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323723 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-trusted-ca-bundle\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.323958 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323740 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-service-ca\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.323958 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-oauth-config\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.323958 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323944 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-oauth-serving-cert\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.324183 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.323973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-serving-cert\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.324536 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.324513 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-service-ca\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.324600 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.324549 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-console-config\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.324699 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.324650 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-oauth-serving-cert\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.324699 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.324671 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-trusted-ca-bundle\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.326528 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.326508 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-serving-cert\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.326621 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.326564 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-oauth-config\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.334391 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.334371 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2jtr\" (UniqueName: \"kubernetes.io/projected/af09956d-eb6f-4497-b917-e9f20df02c77-kube-api-access-x2jtr\") pod \"console-86db4db4fb-xfwbw\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.341819 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.341797 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:29.441684 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.441651 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:29.483745 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.483669 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:29.612209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.612030 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86db4db4fb-xfwbw"] Jul 24 13:51:29.614695 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:51:29.614671 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf09956d_eb6f_4497_b917_e9f20df02c77.slice/crio-0058b134adcc75b9ad847e19fe123a6837bac53ba23557d491b535b885b6b8a7 WatchSource:0}: Error finding container 0058b134adcc75b9ad847e19fe123a6837bac53ba23557d491b535b885b6b8a7: Status 404 returned error can't find the container with id 0058b134adcc75b9ad847e19fe123a6837bac53ba23557d491b535b885b6b8a7 Jul 24 13:51:29.959848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.959813 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:29.997605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.997566 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ql85p" event={"ID":"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5","Type":"ContainerStarted","Data":"97de1edfcec0e88c82a9ce007a8ee9dbb8fef0a1eb5adb470865d22c4ce17c5d"} Jul 24 13:51:29.997788 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.997637 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ql85p" event={"ID":"3c5e9d55-41ee-4f12-ae25-74947f9bc6f5","Type":"ContainerStarted","Data":"b66a66b39cbfafaeeb363765d08943e85ab87e8584b2552a5e9b2dbd4e889ff8"} Jul 24 13:51:29.999145 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.999116 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86db4db4fb-xfwbw" event={"ID":"af09956d-eb6f-4497-b917-e9f20df02c77","Type":"ContainerStarted","Data":"e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1"} Jul 24 13:51:29.999256 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:29.999164 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86db4db4fb-xfwbw" event={"ID":"af09956d-eb6f-4497-b917-e9f20df02c77","Type":"ContainerStarted","Data":"0058b134adcc75b9ad847e19fe123a6837bac53ba23557d491b535b885b6b8a7"} Jul 24 13:51:30.020317 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:30.020268 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ql85p" podStartSLOduration=375.99053657 podStartE2EDuration="6m17.020251964s" totalImagesPullingTime="1.029715394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:51:30.019320608 +0000 UTC m=+376.814663339" watchObservedRunningTime="2026-07-24 13:51:30.020251964 +0000 UTC m=+376.815594693" Jul 24 13:51:30.157335 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:30.157302 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:30.258248 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:30.258169 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:35.414522 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:35.414489 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:35.477209 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:35.477172 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:36.776222 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:36.776190 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:36.842162 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:36.842127 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:39.484062 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:39.484022 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:39.484452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:39.484119 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:39.488845 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:39.488820 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:39.510965 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:39.510905 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86db4db4fb-xfwbw" podStartSLOduration=10.510889283000001 podStartE2EDuration="10.510889283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:51:30.048208706 +0000 UTC m=+376.843551447" watchObservedRunningTime="2026-07-24 13:51:39.510889283 +0000 UTC m=+386.306232013" Jul 24 13:51:40.042664 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:40.042638 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:51:40.088827 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:40.088796 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db59c56b8-ww9wd"] Jul 24 13:51:40.217169 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:40.217139 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:40.314660 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:40.314589 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:40.371250 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:40.371221 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:55.745755 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:55.745708 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:51:55.847005 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:51:55.846951 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:05.108693 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.108657 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6db59c56b8-ww9wd" podUID="00a7fb42-6fd7-4336-94f9-34c7f84cdd70" containerName="console" containerID="cri-o://ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0" gracePeriod=15 Jul 24 13:52:05.356291 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.356265 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:52:05.497916 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.497823 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-config\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.497916 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.497880 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whmkx\" (UniqueName: \"kubernetes.io/projected/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-kube-api-access-whmkx\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.498133 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.497932 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-oauth-config\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.498133 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.497956 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-oauth-serving-cert\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.498133 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498019 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-service-ca\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.498133 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498052 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-trusted-ca-bundle\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.498133 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498078 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-serving-cert\") pod \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\" (UID: \"00a7fb42-6fd7-4336-94f9-34c7f84cdd70\") " Jul 24 13:52:05.498460 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498412 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:52:05.498575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498528 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-service-ca" (OuterVolumeSpecName: "service-ca") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:52:05.498575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498544 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-config" (OuterVolumeSpecName: "console-config") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:52:05.498575 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.498557 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:52:05.500486 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.500460 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:52:05.500688 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.500671 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:52:05.500752 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.500698 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-kube-api-access-whmkx" (OuterVolumeSpecName: "kube-api-access-whmkx") pod "00a7fb42-6fd7-4336-94f9-34c7f84cdd70" (UID: "00a7fb42-6fd7-4336-94f9-34c7f84cdd70"). InnerVolumeSpecName "kube-api-access-whmkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:52:05.598950 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.598914 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.598950 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.598942 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whmkx\" (UniqueName: \"kubernetes.io/projected/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-kube-api-access-whmkx\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.598950 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.598951 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-oauth-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.598950 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.598960 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-oauth-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.599224 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.598972 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-service-ca\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.599224 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.598998 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-trusted-ca-bundle\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.599224 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.599006 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00a7fb42-6fd7-4336-94f9-34c7f84cdd70-console-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:52:05.669354 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:05.669269 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:06.129896 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.129861 2555 generic.go:356] "Generic (PLEG): container finished" podID="00a7fb42-6fd7-4336-94f9-34c7f84cdd70" containerID="ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0" exitCode=0 Jul 24 13:52:06.130376 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.129902 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db59c56b8-ww9wd" event={"ID":"00a7fb42-6fd7-4336-94f9-34c7f84cdd70","Type":"ContainerDied","Data":"ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0"} Jul 24 13:52:06.130376 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.129913 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db59c56b8-ww9wd" Jul 24 13:52:06.130376 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.129954 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db59c56b8-ww9wd" event={"ID":"00a7fb42-6fd7-4336-94f9-34c7f84cdd70","Type":"ContainerDied","Data":"751ada6331565c345fd6e285b7229c019d652f0cab18c9b0154cc8844b1eabf3"} Jul 24 13:52:06.130376 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.130006 2555 scope.go:117] "RemoveContainer" containerID="ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0" Jul 24 13:52:06.138320 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.138302 2555 scope.go:117] "RemoveContainer" containerID="ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0" Jul 24 13:52:06.143009 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:52:06.138581 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0\": container with ID starting with ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0 not found: ID does not exist" containerID="ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0" Jul 24 13:52:06.143009 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.138942 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0"} err="failed to get container status \"ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0\": rpc error: code = NotFound desc = could not find container \"ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0\": container with ID starting with ba7787ee8f11c5e9eb0441c4c805d08077e45a27cc75a2d525924eb6987b1ae0 not found: ID does not exist" Jul 24 13:52:06.146849 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.146821 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db59c56b8-ww9wd"] Jul 24 13:52:06.150248 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:06.150221 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db59c56b8-ww9wd"] Jul 24 13:52:07.748290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:07.748258 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00a7fb42-6fd7-4336-94f9-34c7f84cdd70" path="/var/lib/kubelet/pods/00a7fb42-6fd7-4336-94f9-34c7f84cdd70/volumes" Jul 24 13:52:35.667441 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:35.667362 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:35.724952 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:35.724919 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:42.130229 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.130190 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bfccfc68f-8ndxv"] Jul 24 13:52:42.130667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.130459 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00a7fb42-6fd7-4336-94f9-34c7f84cdd70" containerName="console" Jul 24 13:52:42.130667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.130470 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="00a7fb42-6fd7-4336-94f9-34c7f84cdd70" containerName="console" Jul 24 13:52:42.130667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.130531 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="00a7fb42-6fd7-4336-94f9-34c7f84cdd70" containerName="console" Jul 24 13:52:42.132422 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.132402 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.149498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.149468 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bfccfc68f-8ndxv"] Jul 24 13:52:42.184456 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-service-ca\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.184456 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184462 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqhtg\" (UniqueName: \"kubernetes.io/projected/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-kube-api-access-kqhtg\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.184662 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-oauth-serving-cert\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.184662 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184552 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-config\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.184662 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184587 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-trusted-ca-bundle\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.184662 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-serving-cert\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.184789 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.184692 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-oauth-config\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.189101 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.189078 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:42.245053 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.245019 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:42.285955 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.285919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-oauth-serving-cert\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.285955 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.285955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-config\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286195 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286015 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-trusted-ca-bundle\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286195 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286044 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-serving-cert\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286195 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-oauth-config\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286318 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-service-ca\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286318 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286251 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqhtg\" (UniqueName: \"kubernetes.io/projected/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-kube-api-access-kqhtg\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286805 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-config\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286819 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-oauth-serving-cert\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.286883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.286918 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-trusted-ca-bundle\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.287208 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.287032 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-service-ca\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.288654 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.288625 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-serving-cert\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.288654 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.288645 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-oauth-config\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.294935 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.294913 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqhtg\" (UniqueName: \"kubernetes.io/projected/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-kube-api-access-kqhtg\") pod \"console-bfccfc68f-8ndxv\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.441450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.441364 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:42.584807 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:42.584775 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bfccfc68f-8ndxv"] Jul 24 13:52:42.588419 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:52:42.588377 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0422a600_5b0e_44be_a9ef_aaf3b6a379c6.slice/crio-4c6af0202a3803bd0e123b9881d08a89c445dc52d120bf0a749438071ff01350 WatchSource:0}: Error finding container 4c6af0202a3803bd0e123b9881d08a89c445dc52d120bf0a749438071ff01350: Status 404 returned error can't find the container with id 4c6af0202a3803bd0e123b9881d08a89c445dc52d120bf0a749438071ff01350 Jul 24 13:52:43.258819 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:43.258777 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfccfc68f-8ndxv" event={"ID":"0422a600-5b0e-44be-a9ef-aaf3b6a379c6","Type":"ContainerStarted","Data":"de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e"} Jul 24 13:52:43.259320 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:43.258830 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfccfc68f-8ndxv" event={"ID":"0422a600-5b0e-44be-a9ef-aaf3b6a379c6","Type":"ContainerStarted","Data":"4c6af0202a3803bd0e123b9881d08a89c445dc52d120bf0a749438071ff01350"} Jul 24 13:52:43.277266 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:43.277220 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bfccfc68f-8ndxv" podStartSLOduration=1.277204683 podStartE2EDuration="1.277204683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:52:43.275913454 +0000 UTC m=+450.071256184" watchObservedRunningTime="2026-07-24 13:52:43.277204683 +0000 UTC m=+450.072547477" Jul 24 13:52:47.638228 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:47.638183 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:47.692729 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:47.692693 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:52.441690 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:52.441652 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:52.442126 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:52.441736 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:52.446666 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:52.446642 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:53.297536 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:53.297510 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:52:53.343282 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:53.343247 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86db4db4fb-xfwbw"] Jul 24 13:52:53.448366 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:53.448329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:53.501761 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:53.501732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:53.558096 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:53.558011 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:56.638548 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:56.638513 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:56.739763 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:56.739729 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:52:57.004448 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:52:57.004366 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:18.367308 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.367227 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86db4db4fb-xfwbw" podUID="af09956d-eb6f-4497-b917-e9f20df02c77" containerName="console" containerID="cri-o://e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1" gracePeriod=15 Jul 24 13:53:18.607237 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.607214 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:53:18.780714 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780631 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2jtr\" (UniqueName: \"kubernetes.io/projected/af09956d-eb6f-4497-b917-e9f20df02c77-kube-api-access-x2jtr\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.780714 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780671 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-serving-cert\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.780714 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780694 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-console-config\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.780714 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780714 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-oauth-config\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.781058 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780768 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-service-ca\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.781058 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780791 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-oauth-serving-cert\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.781058 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.780811 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-trusted-ca-bundle\") pod \"af09956d-eb6f-4497-b917-e9f20df02c77\" (UID: \"af09956d-eb6f-4497-b917-e9f20df02c77\") " Jul 24 13:53:18.781246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.781215 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:53:18.781246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.781233 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-service-ca" (OuterVolumeSpecName: "service-ca") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:53:18.781332 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.781227 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-console-config" (OuterVolumeSpecName: "console-config") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:53:18.781332 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.781292 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:53:18.782955 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.782938 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:53:18.783498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.783478 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:53:18.783498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.783488 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af09956d-eb6f-4497-b917-e9f20df02c77-kube-api-access-x2jtr" (OuterVolumeSpecName: "kube-api-access-x2jtr") pod "af09956d-eb6f-4497-b917-e9f20df02c77" (UID: "af09956d-eb6f-4497-b917-e9f20df02c77"). InnerVolumeSpecName "kube-api-access-x2jtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:53:18.882295 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882257 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2jtr\" (UniqueName: \"kubernetes.io/projected/af09956d-eb6f-4497-b917-e9f20df02c77-kube-api-access-x2jtr\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:18.882295 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882286 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:18.882295 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882295 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-console-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:18.882295 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882304 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af09956d-eb6f-4497-b917-e9f20df02c77-console-oauth-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:18.882558 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882313 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-service-ca\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:18.882558 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882322 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-oauth-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:18.882558 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:18.882330 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af09956d-eb6f-4497-b917-e9f20df02c77-trusted-ca-bundle\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:53:19.383341 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.383311 2555 generic.go:356] "Generic (PLEG): container finished" podID="af09956d-eb6f-4497-b917-e9f20df02c77" containerID="e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1" exitCode=0 Jul 24 13:53:19.383785 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.383356 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86db4db4fb-xfwbw" Jul 24 13:53:19.383785 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.383375 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86db4db4fb-xfwbw" event={"ID":"af09956d-eb6f-4497-b917-e9f20df02c77","Type":"ContainerDied","Data":"e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1"} Jul 24 13:53:19.383785 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.383415 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86db4db4fb-xfwbw" event={"ID":"af09956d-eb6f-4497-b917-e9f20df02c77","Type":"ContainerDied","Data":"0058b134adcc75b9ad847e19fe123a6837bac53ba23557d491b535b885b6b8a7"} Jul 24 13:53:19.383785 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.383434 2555 scope.go:117] "RemoveContainer" containerID="e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1" Jul 24 13:53:19.392570 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.392554 2555 scope.go:117] "RemoveContainer" containerID="e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1" Jul 24 13:53:19.392849 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:53:19.392830 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1\": container with ID starting with e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1 not found: ID does not exist" containerID="e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1" Jul 24 13:53:19.392915 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.392855 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1"} err="failed to get container status \"e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1\": rpc error: code = NotFound desc = could not find container \"e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1\": container with ID starting with e5f653e67dd72e221db4e8fa54cfc2e3f888ee8aa0976e5508261a7476498da1 not found: ID does not exist" Jul 24 13:53:19.404274 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.404241 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86db4db4fb-xfwbw"] Jul 24 13:53:19.406033 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.406010 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86db4db4fb-xfwbw"] Jul 24 13:53:19.748612 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:19.748532 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af09956d-eb6f-4497-b917-e9f20df02c77" path="/var/lib/kubelet/pods/af09956d-eb6f-4497-b917-e9f20df02c77/volumes" Jul 24 13:53:23.664522 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:23.664490 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:23.763037 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:23.763003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:28.714145 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:28.714116 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:28.773337 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:28.773301 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:45.289661 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:45.289612 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:46.098083 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:46.098050 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:47.230858 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.230821 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw"] Jul 24 13:53:47.231357 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.231101 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af09956d-eb6f-4497-b917-e9f20df02c77" containerName="console" Jul 24 13:53:47.231357 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.231113 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="af09956d-eb6f-4497-b917-e9f20df02c77" containerName="console" Jul 24 13:53:47.231357 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.231165 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="af09956d-eb6f-4497-b917-e9f20df02c77" containerName="console" Jul 24 13:53:47.233884 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.233862 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.236430 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.236404 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:53:47.236567 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.236441 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 24 13:53:47.236567 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.236407 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-k5v6k\"" Jul 24 13:53:47.247257 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.247228 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw"] Jul 24 13:53:47.282917 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.282872 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjcw6\" (UniqueName: \"kubernetes.io/projected/0be671b8-a709-4375-840a-e71d7bd72cf4-kube-api-access-sjcw6\") pod \"cert-manager-operator-controller-manager-55c444bf86-4ffkw\" (UID: \"0be671b8-a709-4375-840a-e71d7bd72cf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.282917 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.282924 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0be671b8-a709-4375-840a-e71d7bd72cf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4ffkw\" (UID: \"0be671b8-a709-4375-840a-e71d7bd72cf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.384276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.384233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjcw6\" (UniqueName: \"kubernetes.io/projected/0be671b8-a709-4375-840a-e71d7bd72cf4-kube-api-access-sjcw6\") pod \"cert-manager-operator-controller-manager-55c444bf86-4ffkw\" (UID: \"0be671b8-a709-4375-840a-e71d7bd72cf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.384276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.384282 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0be671b8-a709-4375-840a-e71d7bd72cf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4ffkw\" (UID: \"0be671b8-a709-4375-840a-e71d7bd72cf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.384624 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.384606 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0be671b8-a709-4375-840a-e71d7bd72cf4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4ffkw\" (UID: \"0be671b8-a709-4375-840a-e71d7bd72cf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.392649 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.392623 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjcw6\" (UniqueName: \"kubernetes.io/projected/0be671b8-a709-4375-840a-e71d7bd72cf4-kube-api-access-sjcw6\") pod \"cert-manager-operator-controller-manager-55c444bf86-4ffkw\" (UID: \"0be671b8-a709-4375-840a-e71d7bd72cf4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.543323 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.543291 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" Jul 24 13:53:47.683749 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:47.683719 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw"] Jul 24 13:53:47.687132 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:53:47.687103 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be671b8_a709_4375_840a_e71d7bd72cf4.slice/crio-1785e8f0ade70a0e8b9cb68d02c41a5d8ed73d354b2702c4b987036e0b2eafb8 WatchSource:0}: Error finding container 1785e8f0ade70a0e8b9cb68d02c41a5d8ed73d354b2702c4b987036e0b2eafb8: Status 404 returned error can't find the container with id 1785e8f0ade70a0e8b9cb68d02c41a5d8ed73d354b2702c4b987036e0b2eafb8 Jul 24 13:53:48.492681 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:48.492634 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" event={"ID":"0be671b8-a709-4375-840a-e71d7bd72cf4","Type":"ContainerStarted","Data":"1785e8f0ade70a0e8b9cb68d02c41a5d8ed73d354b2702c4b987036e0b2eafb8"} Jul 24 13:53:48.915203 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:48.915115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:49.015804 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:49.015755 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:50.504248 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:50.504133 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" event={"ID":"0be671b8-a709-4375-840a-e71d7bd72cf4","Type":"ContainerStarted","Data":"2cba210a70c1f1cb2be52c252da1b53415ac11e883921c60c6f13aec92303df9"} Jul 24 13:53:50.527272 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:50.527205 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4ffkw" podStartSLOduration=1.00694516 podStartE2EDuration="3.527183973s" totalImagesPullingTime="2.520238813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:53:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:53:50.525573774 +0000 UTC m=+517.320916507" watchObservedRunningTime="2026-07-24 13:53:50.527183973 +0000 UTC m=+517.322526705" Jul 24 13:53:50.932611 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:50.932546 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:51.073504 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:51.073471 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:51.194027 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:51.193913 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:53.421629 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:53.421592 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:53.830850 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:53.830809 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:56.281775 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.281740 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ccpml"] Jul 24 13:53:56.284949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.284931 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.287725 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.287695 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 13:53:56.287725 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.287714 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 13:53:56.287932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.287781 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2nb2s\"" Jul 24 13:53:56.295136 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.295106 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ccpml"] Jul 24 13:53:56.356215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.356176 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b2a5bed-be63-4134-84de-66213121c374-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ccpml\" (UID: \"2b2a5bed-be63-4134-84de-66213121c374\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.356398 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.356232 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmpss\" (UniqueName: \"kubernetes.io/projected/2b2a5bed-be63-4134-84de-66213121c374-kube-api-access-rmpss\") pod \"cert-manager-cainjector-559cdd8c87-ccpml\" (UID: \"2b2a5bed-be63-4134-84de-66213121c374\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.457477 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.457445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmpss\" (UniqueName: \"kubernetes.io/projected/2b2a5bed-be63-4134-84de-66213121c374-kube-api-access-rmpss\") pod \"cert-manager-cainjector-559cdd8c87-ccpml\" (UID: \"2b2a5bed-be63-4134-84de-66213121c374\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.457603 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.457507 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b2a5bed-be63-4134-84de-66213121c374-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ccpml\" (UID: \"2b2a5bed-be63-4134-84de-66213121c374\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.466037 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.466008 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b2a5bed-be63-4134-84de-66213121c374-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ccpml\" (UID: \"2b2a5bed-be63-4134-84de-66213121c374\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.466215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.466192 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmpss\" (UniqueName: \"kubernetes.io/projected/2b2a5bed-be63-4134-84de-66213121c374-kube-api-access-rmpss\") pod \"cert-manager-cainjector-559cdd8c87-ccpml\" (UID: \"2b2a5bed-be63-4134-84de-66213121c374\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.598279 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.598243 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" Jul 24 13:53:56.732377 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:56.732342 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ccpml"] Jul 24 13:53:56.735286 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:53:56.735257 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b2a5bed_be63_4134_84de_66213121c374.slice/crio-5836bc2399f167dcb9a34666416c44f877e352187839c94ee1fa056d8be43650 WatchSource:0}: Error finding container 5836bc2399f167dcb9a34666416c44f877e352187839c94ee1fa056d8be43650: Status 404 returned error can't find the container with id 5836bc2399f167dcb9a34666416c44f877e352187839c94ee1fa056d8be43650 Jul 24 13:53:57.534935 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:57.534892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" event={"ID":"2b2a5bed-be63-4134-84de-66213121c374","Type":"ContainerStarted","Data":"5836bc2399f167dcb9a34666416c44f877e352187839c94ee1fa056d8be43650"} Jul 24 13:53:58.920885 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:58.920844 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:53:59.013583 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:53:59.013547 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:00.457313 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:00.457285 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:00.549884 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:00.549851 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" event={"ID":"2b2a5bed-be63-4134-84de-66213121c374","Type":"ContainerStarted","Data":"99383ffc394097ca3e0ad65034f473d03d888686f0d6d607b5b087f9495ee9ce"} Jul 24 13:54:00.554035 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:00.553973 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:00.565936 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:00.565884 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ccpml" podStartSLOduration=1.6303344389999999 podStartE2EDuration="4.56586704s" totalImagesPullingTime="2.935532601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:00.565411808 +0000 UTC m=+527.360754549" watchObservedRunningTime="2026-07-24 13:54:00.56586704 +0000 UTC m=+527.361209769" Jul 24 13:54:01.024233 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:01.024198 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:04.592770 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:04.592683 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:04.693028 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:04.692972 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:06.585850 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:06.585818 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:22.507838 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:22.507796 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:23.545563 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.545528 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr"] Jul 24 13:54:23.555056 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.555022 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.557696 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.557663 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 13:54:23.557841 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.557772 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ft8zn\"" Jul 24 13:54:23.557841 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.557796 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 13:54:23.557841 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.557823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 13:54:23.558029 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.557825 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 13:54:23.564608 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.564582 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr"] Jul 24 13:54:23.674033 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.673999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b3b06285-f2fb-4be5-b911-20c6ac3bde46-manifests\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.674033 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.674036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3b06285-f2fb-4be5-b911-20c6ac3bde46-tmp\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.674297 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.674061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b3b06285-f2fb-4be5-b911-20c6ac3bde46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.674297 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.674082 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79cf2\" (UniqueName: \"kubernetes.io/projected/b3b06285-f2fb-4be5-b911-20c6ac3bde46-kube-api-access-79cf2\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.674297 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.674180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3b06285-f2fb-4be5-b911-20c6ac3bde46-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.708349 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.708309 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:23.775409 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.775375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b3b06285-f2fb-4be5-b911-20c6ac3bde46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.775409 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.775413 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79cf2\" (UniqueName: \"kubernetes.io/projected/b3b06285-f2fb-4be5-b911-20c6ac3bde46-kube-api-access-79cf2\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.775626 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.775596 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3b06285-f2fb-4be5-b911-20c6ac3bde46-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.775691 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.775669 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b3b06285-f2fb-4be5-b911-20c6ac3bde46-manifests\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.775830 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.775699 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3b06285-f2fb-4be5-b911-20c6ac3bde46-tmp\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.776030 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.775964 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b3b06285-f2fb-4be5-b911-20c6ac3bde46-manifests\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.777977 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.777949 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3b06285-f2fb-4be5-b911-20c6ac3bde46-tmp\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.778135 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.778118 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b3b06285-f2fb-4be5-b911-20c6ac3bde46-webhook-cert\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.778192 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.778178 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b3b06285-f2fb-4be5-b911-20c6ac3bde46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.784354 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.784324 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79cf2\" (UniqueName: \"kubernetes.io/projected/b3b06285-f2fb-4be5-b911-20c6ac3bde46-kube-api-access-79cf2\") pod \"opendatahub-operator-controller-manager-6cd96dfd5f-slsbr\" (UID: \"b3b06285-f2fb-4be5-b911-20c6ac3bde46\") " pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.805370 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.805287 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:23.865508 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.865470 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:23.907176 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:23.907141 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:24.009904 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:24.009861 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr"] Jul 24 13:54:24.013217 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:54:24.013188 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b06285_f2fb_4be5_b911_20c6ac3bde46.slice/crio-e516329680706709143a174335886e810868b592d0f3597abc20e96c96e33e7f WatchSource:0}: Error finding container e516329680706709143a174335886e810868b592d0f3597abc20e96c96e33e7f: Status 404 returned error can't find the container with id e516329680706709143a174335886e810868b592d0f3597abc20e96c96e33e7f Jul 24 13:54:24.014583 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:24.014541 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:24.106611 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:24.106526 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:24.208267 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:24.208230 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:24.643556 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:24.643514 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" event={"ID":"b3b06285-f2fb-4be5-b911-20c6ac3bde46","Type":"ContainerStarted","Data":"e516329680706709143a174335886e810868b592d0f3597abc20e96c96e33e7f"} Jul 24 13:54:27.345307 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:27.345268 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:27.440646 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:27.440610 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:27.659172 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:27.659075 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" event={"ID":"b3b06285-f2fb-4be5-b911-20c6ac3bde46","Type":"ContainerStarted","Data":"2db7d90c7a0e9ed12cbc7ca967a60ba2ae427411ce2f894e8ff74b32327fc355"} Jul 24 13:54:28.664290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:28.664250 2555 generic.go:356] "Generic (PLEG): container finished" podID="b3b06285-f2fb-4be5-b911-20c6ac3bde46" containerID="2db7d90c7a0e9ed12cbc7ca967a60ba2ae427411ce2f894e8ff74b32327fc355" exitCode=0 Jul 24 13:54:28.664776 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:28.664307 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" event={"ID":"b3b06285-f2fb-4be5-b911-20c6ac3bde46","Type":"ContainerDied","Data":"2db7d90c7a0e9ed12cbc7ca967a60ba2ae427411ce2f894e8ff74b32327fc355"} Jul 24 13:54:29.670365 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:29.670334 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" event={"ID":"b3b06285-f2fb-4be5-b911-20c6ac3bde46","Type":"ContainerStarted","Data":"0b7043cfaa52e62632bf19c45d40b0b9a2b565fc7423a985e79b22fa4d16b503"} Jul 24 13:54:29.670752 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:29.670476 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:29.693500 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:29.693444 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" podStartSLOduration=1.886535742 podStartE2EDuration="6.693425049s" totalImagesPullingTime="3.801623434s" totalInitContainerRuntime="1.005265873s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:54:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:29.693034484 +0000 UTC m=+556.488377218" watchObservedRunningTime="2026-07-24 13:54:29.693425049 +0000 UTC m=+556.488767780" Jul 24 13:54:35.659423 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:35.659382 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:35.765192 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:35.765148 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:40.676267 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:40.676237 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6cd96dfd5f-slsbr" Jul 24 13:54:42.700719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:42.700687 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:42.820133 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:42.820104 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:43.404068 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.404027 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b9kt9"] Jul 24 13:54:43.407245 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.407218 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:43.410411 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.410395 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 24 13:54:43.410631 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.410617 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-crb47\"" Jul 24 13:54:43.422635 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.422611 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b9kt9"] Jul 24 13:54:43.473724 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.473691 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:43.546207 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.546158 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hd8d\" (UniqueName: \"kubernetes.io/projected/61c5caff-6c6b-4082-9da9-0198004dc5c2-kube-api-access-5hd8d\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:43.546394 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.546233 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c5caff-6c6b-4082-9da9-0198004dc5c2-cert\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:43.617265 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.617234 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:43.646644 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.646607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hd8d\" (UniqueName: \"kubernetes.io/projected/61c5caff-6c6b-4082-9da9-0198004dc5c2-kube-api-access-5hd8d\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:43.646829 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.646664 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c5caff-6c6b-4082-9da9-0198004dc5c2-cert\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:43.646829 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:54:43.646764 2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 24 13:54:43.646927 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:54:43.646832 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61c5caff-6c6b-4082-9da9-0198004dc5c2-cert podName:61c5caff-6c6b-4082-9da9-0198004dc5c2 nodeName:}" failed. No retries permitted until 2026-07-24 13:54:44.146812938 +0000 UTC m=+570.942155663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/61c5caff-6c6b-4082-9da9-0198004dc5c2-cert") pod "odh-model-controller-7998c67dc4-b9kt9" (UID: "61c5caff-6c6b-4082-9da9-0198004dc5c2") : secret "odh-model-controller-webhook-cert" not found Jul 24 13:54:43.663704 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.663630 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hd8d\" (UniqueName: \"kubernetes.io/projected/61c5caff-6c6b-4082-9da9-0198004dc5c2-kube-api-access-5hd8d\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:43.713584 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.713547 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:43.813771 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:43.813732 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:44.150476 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.150423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c5caff-6c6b-4082-9da9-0198004dc5c2-cert\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:44.153185 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.153161 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/61c5caff-6c6b-4082-9da9-0198004dc5c2-cert\") pod \"odh-model-controller-7998c67dc4-b9kt9\" (UID: \"61c5caff-6c6b-4082-9da9-0198004dc5c2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:44.156140 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.156112 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:44.202910 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.202874 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nkmxv"] Jul 24 13:54:44.206107 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.206090 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.208612 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.208593 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 24 13:54:44.208766 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.208751 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-x5m5f\"" Jul 24 13:54:44.214274 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.214251 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nkmxv"] Jul 24 13:54:44.317673 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.317635 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:44.352885 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.352847 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4cb4\" (UniqueName: \"kubernetes.io/projected/bc0b970c-0087-47b7-906e-bdb81fb53851-kube-api-access-z4cb4\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.353091 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.352967 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0b970c-0087-47b7-906e-bdb81fb53851-tls-certs\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.450521 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.450491 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b9kt9"] Jul 24 13:54:44.452580 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:54:44.452551 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c5caff_6c6b_4082_9da9_0198004dc5c2.slice/crio-1356790a31081e75621289b58de9b472e3cd1f33d034d9f67ff92d79330beac0 WatchSource:0}: Error finding container 1356790a31081e75621289b58de9b472e3cd1f33d034d9f67ff92d79330beac0: Status 404 returned error can't find the container with id 1356790a31081e75621289b58de9b472e3cd1f33d034d9f67ff92d79330beac0 Jul 24 13:54:44.453457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.453428 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cb4\" (UniqueName: \"kubernetes.io/projected/bc0b970c-0087-47b7-906e-bdb81fb53851-kube-api-access-z4cb4\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.453583 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.453551 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0b970c-0087-47b7-906e-bdb81fb53851-tls-certs\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.453675 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:54:44.453655 2555 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 24 13:54:44.453801 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:54:44.453721 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc0b970c-0087-47b7-906e-bdb81fb53851-tls-certs podName:bc0b970c-0087-47b7-906e-bdb81fb53851 nodeName:}" failed. No retries permitted until 2026-07-24 13:54:44.953702987 +0000 UTC m=+571.749045694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bc0b970c-0087-47b7-906e-bdb81fb53851-tls-certs") pod "model-serving-api-54d89bd5fc-nkmxv" (UID: "bc0b970c-0087-47b7-906e-bdb81fb53851") : secret "model-serving-api-tls" not found Jul 24 13:54:44.461835 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.461812 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4cb4\" (UniqueName: \"kubernetes.io/projected/bc0b970c-0087-47b7-906e-bdb81fb53851-kube-api-access-z4cb4\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.616339 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.616291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:44.733494 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.733402 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" event={"ID":"61c5caff-6c6b-4082-9da9-0198004dc5c2","Type":"ContainerStarted","Data":"1356790a31081e75621289b58de9b472e3cd1f33d034d9f67ff92d79330beac0"} Jul 24 13:54:44.782998 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.782954 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:44.959716 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.959671 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0b970c-0087-47b7-906e-bdb81fb53851-tls-certs\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:44.962280 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:44.962255 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bc0b970c-0087-47b7-906e-bdb81fb53851-tls-certs\") pod \"model-serving-api-54d89bd5fc-nkmxv\" (UID: \"bc0b970c-0087-47b7-906e-bdb81fb53851\") " pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:45.118802 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:45.118757 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:45.285143 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:45.285104 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nkmxv"] Jul 24 13:54:45.739467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:45.739430 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" event={"ID":"bc0b970c-0087-47b7-906e-bdb81fb53851","Type":"ContainerStarted","Data":"7fdc76c34a7e653c309a2854e4e749a6fea575ca6d3b7fafadb1b99b3bd74202"} Jul 24 13:54:47.750175 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:47.750140 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" event={"ID":"61c5caff-6c6b-4082-9da9-0198004dc5c2","Type":"ContainerStarted","Data":"7eea94be05928f0ac397dc0566283a1b570c485f823a194326a6673ff066bd2a"} Jul 24 13:54:47.750630 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:47.750212 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:54:47.766541 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:47.766480 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" podStartSLOduration=2.108095404 podStartE2EDuration="4.766456735s" totalImagesPullingTime="2.658361331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:47.765789012 +0000 UTC m=+574.561131742" watchObservedRunningTime="2026-07-24 13:54:47.766456735 +0000 UTC m=+574.561799466" Jul 24 13:54:48.755527 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:48.755489 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" event={"ID":"bc0b970c-0087-47b7-906e-bdb81fb53851","Type":"ContainerStarted","Data":"185db28c97352370efdde7ee7eaa5be1bf794caf7b7c796fd55e4ebf9761b2cc"} Jul 24 13:54:48.772637 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:48.772582 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" podStartSLOduration=1.995530313 podStartE2EDuration="4.772565412s" totalImagesPullingTime="2.777035099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:48.771632213 +0000 UTC m=+575.566974942" watchObservedRunningTime="2026-07-24 13:54:48.772565412 +0000 UTC m=+575.567908141" Jul 24 13:54:49.759728 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:49.759691 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:54:50.537663 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.537625 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz"] Jul 24 13:54:50.540976 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.540953 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.544834 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.544809 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 13:54:50.545162 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.545129 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 13:54:50.545245 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.545205 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:54:50.545426 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.545409 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fpjcj\"" Jul 24 13:54:50.545477 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.545422 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 13:54:50.545477 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.545419 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 13:54:50.552174 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.552151 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz"] Jul 24 13:54:50.616677 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.616641 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/082170f1-beec-444c-ba1b-3d67b930a43b-manager-config\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.616848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.616690 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/082170f1-beec-444c-ba1b-3d67b930a43b-metrics-cert\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.616848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.616706 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/082170f1-beec-444c-ba1b-3d67b930a43b-cert\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.616848 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.616741 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkd7b\" (UniqueName: \"kubernetes.io/projected/082170f1-beec-444c-ba1b-3d67b930a43b-kube-api-access-qkd7b\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.718086 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.718046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/082170f1-beec-444c-ba1b-3d67b930a43b-metrics-cert\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.718086 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.718084 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/082170f1-beec-444c-ba1b-3d67b930a43b-cert\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.718326 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.718126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkd7b\" (UniqueName: \"kubernetes.io/projected/082170f1-beec-444c-ba1b-3d67b930a43b-kube-api-access-qkd7b\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.718326 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.718158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/082170f1-beec-444c-ba1b-3d67b930a43b-manager-config\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.718802 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.718782 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/082170f1-beec-444c-ba1b-3d67b930a43b-manager-config\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.720627 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.720603 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/082170f1-beec-444c-ba1b-3d67b930a43b-metrics-cert\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.720747 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.720725 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/082170f1-beec-444c-ba1b-3d67b930a43b-cert\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.727242 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.727218 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkd7b\" (UniqueName: \"kubernetes.io/projected/082170f1-beec-444c-ba1b-3d67b930a43b-kube-api-access-qkd7b\") pod \"lws-controller-manager-84df8c9cc-h9njz\" (UID: \"082170f1-beec-444c-ba1b-3d67b930a43b\") " pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.850768 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.850728 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:50.995301 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:50.993455 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz"] Jul 24 13:54:50.998404 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:54:50.998369 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082170f1_beec_444c_ba1b_3d67b930a43b.slice/crio-3c4d2d95db12758913cdbd9100c451bec43babf81a1fda19ae69cb9628119eaf WatchSource:0}: Error finding container 3c4d2d95db12758913cdbd9100c451bec43babf81a1fda19ae69cb9628119eaf: Status 404 returned error can't find the container with id 3c4d2d95db12758913cdbd9100c451bec43babf81a1fda19ae69cb9628119eaf Jul 24 13:54:51.769032 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:51.768972 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" event={"ID":"082170f1-beec-444c-ba1b-3d67b930a43b","Type":"ContainerStarted","Data":"3c4d2d95db12758913cdbd9100c451bec43babf81a1fda19ae69cb9628119eaf"} Jul 24 13:54:52.758151 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:52.758108 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:52.862885 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:52.862852 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:52.973857 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:52.973824 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:53.088102 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:53.088062 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:53.709734 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:53.709687 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:53.781002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:53.780910 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" event={"ID":"082170f1-beec-444c-ba1b-3d67b930a43b","Type":"ContainerStarted","Data":"3f26356c3165b095a6724e2804cc8290c4988f45619ee35544c1a0555bc85f24"} Jul 24 13:54:53.781612 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:53.781048 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:54:53.804012 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:53.803929 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" podStartSLOduration=1.135762702 podStartE2EDuration="3.803912479s" totalImagesPullingTime="2.668149777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:54:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:53.803457855 +0000 UTC m=+580.598800585" watchObservedRunningTime="2026-07-24 13:54:53.803912479 +0000 UTC m=+580.599255209" Jul 24 13:54:53.875283 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:53.875254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:54.107527 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:54.107493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:54.192833 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:54.192766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:54.293345 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:54.293317 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:55.490164 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:55.490127 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:55.572862 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:55.572833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:56.660659 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:56.660622 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:57.353336 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:57.353302 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:57.448164 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:57.448134 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:54:58.757727 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:54:58.757691 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b9kt9" Jul 24 13:55:00.341766 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.341735 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:00.401941 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.401905 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:00.768930 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.768847 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nkmxv" Jul 24 13:55:00.890491 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.890458 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9"] Jul 24 13:55:00.898082 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.898057 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:00.901059 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.900967 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 13:55:00.901215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.900969 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-t4rhs\"" Jul 24 13:55:00.901215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.901135 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 13:55:00.901215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.901059 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 13:55:00.906820 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.906794 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9"] Jul 24 13:55:00.926077 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:00.926043 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:01.006719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.006675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.006932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.006794 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.006932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.006835 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/941b394c-91cb-4af3-90cb-8ad0751070ab-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.006932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.006886 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9gzd\" (UniqueName: \"kubernetes.io/projected/941b394c-91cb-4af3-90cb-8ad0751070ab-kube-api-access-c9gzd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.006932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.006908 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.007180 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.006941 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.007180 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.007043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.007180 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.007074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.007180 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.007097 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108450 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108665 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108476 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108665 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108509 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/941b394c-91cb-4af3-90cb-8ad0751070ab-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108665 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108557 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9gzd\" (UniqueName: \"kubernetes.io/projected/941b394c-91cb-4af3-90cb-8ad0751070ab-kube-api-access-c9gzd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108825 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108684 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108825 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108744 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108825 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108802 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108966 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.108966 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.108860 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.109111 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.109031 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.109404 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.109347 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.109494 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.109421 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/941b394c-91cb-4af3-90cb-8ad0751070ab-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.109494 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.109429 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.109562 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.109534 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.111434 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.111401 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.111528 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.111429 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.119974 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.119943 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9gzd\" (UniqueName: \"kubernetes.io/projected/941b394c-91cb-4af3-90cb-8ad0751070ab-kube-api-access-c9gzd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.120432 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.120413 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/941b394c-91cb-4af3-90cb-8ad0751070ab-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9\" (UID: \"941b394c-91cb-4af3-90cb-8ad0751070ab\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.141463 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.141429 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:01.210411 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.210378 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:01.241326 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.241294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:01.350569 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.350537 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9"] Jul 24 13:55:01.353954 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:55:01.353927 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod941b394c_91cb_4af3_90cb_8ad0751070ab.slice/crio-f4b154719a47b2f0fa9c57247fbe85dda928921704f441dca6f0afbb98008afd WatchSource:0}: Error finding container f4b154719a47b2f0fa9c57247fbe85dda928921704f441dca6f0afbb98008afd: Status 404 returned error can't find the container with id f4b154719a47b2f0fa9c57247fbe85dda928921704f441dca6f0afbb98008afd Jul 24 13:55:01.821252 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:01.821220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" event={"ID":"941b394c-91cb-4af3-90cb-8ad0751070ab","Type":"ContainerStarted","Data":"f4b154719a47b2f0fa9c57247fbe85dda928921704f441dca6f0afbb98008afd"} Jul 24 13:55:04.787457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:04.787407 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:55:04.787721 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:04.787507 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:55:04.787721 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:04.787538 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:55:04.790838 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:04.790818 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84df8c9cc-h9njz" Jul 24 13:55:05.054206 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:05.054168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:05.110167 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:05.110137 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:05.842106 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:05.842074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" event={"ID":"941b394c-91cb-4af3-90cb-8ad0751070ab","Type":"ContainerStarted","Data":"a73efffed4fc982098917414b7f830fd701f6c0798d45cb3a3281d41457b312a"} Jul 24 13:55:05.862776 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:05.862723 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" podStartSLOduration=2.431388073 podStartE2EDuration="5.862706753s" totalImagesPullingTime="3.43131868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:05.861228233 +0000 UTC m=+592.656570967" watchObservedRunningTime="2026-07-24 13:55:05.862706753 +0000 UTC m=+592.658049482" Jul 24 13:55:06.211101 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:06.211008 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:06.215604 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:06.215579 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:06.846899 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:06.846866 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:06.847954 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:06.847932 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9" Jul 24 13:55:43.670441 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:43.670352 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:44.261965 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.261928 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:44.358437 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.358406 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:44.388725 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.388691 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f6vx4"] Jul 24 13:55:44.394330 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.394302 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:44.396829 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.396799 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jqfvs\"" Jul 24 13:55:44.397010 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.396799 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:55:44.398188 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.398166 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:55:44.400846 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.400792 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f6vx4"] Jul 24 13:55:44.439197 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.439168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:55:44.485457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.485420 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wtwt\" (UniqueName: \"kubernetes.io/projected/72bb2246-854c-4c29-b01f-ac876a0d9514-kube-api-access-4wtwt\") pod \"kuadrant-operator-catalog-f6vx4\" (UID: \"72bb2246-854c-4c29-b01f-ac876a0d9514\") " pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:44.586216 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.586177 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wtwt\" (UniqueName: \"kubernetes.io/projected/72bb2246-854c-4c29-b01f-ac876a0d9514-kube-api-access-4wtwt\") pod \"kuadrant-operator-catalog-f6vx4\" (UID: \"72bb2246-854c-4c29-b01f-ac876a0d9514\") " pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:44.594507 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.594484 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wtwt\" (UniqueName: \"kubernetes.io/projected/72bb2246-854c-4c29-b01f-ac876a0d9514-kube-api-access-4wtwt\") pod \"kuadrant-operator-catalog-f6vx4\" (UID: \"72bb2246-854c-4c29-b01f-ac876a0d9514\") " pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:44.706189 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.706140 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:44.744736 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.744699 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f6vx4"] Jul 24 13:55:44.842492 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.842468 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f6vx4"] Jul 24 13:55:44.844772 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:55:44.844743 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bb2246_854c_4c29_b01f_ac876a0d9514.slice/crio-b6282fd27c833d6286221b7473fd62a640ef3fb30e1abb2bdd8c6959ecda990e WatchSource:0}: Error finding container b6282fd27c833d6286221b7473fd62a640ef3fb30e1abb2bdd8c6959ecda990e: Status 404 returned error can't find the container with id b6282fd27c833d6286221b7473fd62a640ef3fb30e1abb2bdd8c6959ecda990e Jul 24 13:55:44.846574 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.846554 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:55:44.958222 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.958186 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sztsl"] Jul 24 13:55:44.965261 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.965224 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:44.968227 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.968197 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sztsl"] Jul 24 13:55:44.989078 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:44.989044 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cz4z\" (UniqueName: \"kubernetes.io/projected/51ef3fae-305b-4ed6-a06d-54471d352f74-kube-api-access-7cz4z\") pod \"kuadrant-operator-catalog-sztsl\" (UID: \"51ef3fae-305b-4ed6-a06d-54471d352f74\") " pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:45.003819 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:45.003786 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" event={"ID":"72bb2246-854c-4c29-b01f-ac876a0d9514","Type":"ContainerStarted","Data":"b6282fd27c833d6286221b7473fd62a640ef3fb30e1abb2bdd8c6959ecda990e"} Jul 24 13:55:45.089458 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:45.089418 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cz4z\" (UniqueName: \"kubernetes.io/projected/51ef3fae-305b-4ed6-a06d-54471d352f74-kube-api-access-7cz4z\") pod \"kuadrant-operator-catalog-sztsl\" (UID: \"51ef3fae-305b-4ed6-a06d-54471d352f74\") " pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:45.097717 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:45.097660 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cz4z\" (UniqueName: \"kubernetes.io/projected/51ef3fae-305b-4ed6-a06d-54471d352f74-kube-api-access-7cz4z\") pod \"kuadrant-operator-catalog-sztsl\" (UID: \"51ef3fae-305b-4ed6-a06d-54471d352f74\") " pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:45.276702 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:45.276663 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:45.418334 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:45.418302 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sztsl"] Jul 24 13:55:45.445823 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:55:45.445790 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ef3fae_305b_4ed6_a06d_54471d352f74.slice/crio-7068f871226c52315edfa833f8e67d9e3fa6c2a43a0ab2be0720ae06cbd60cc5 WatchSource:0}: Error finding container 7068f871226c52315edfa833f8e67d9e3fa6c2a43a0ab2be0720ae06cbd60cc5: Status 404 returned error can't find the container with id 7068f871226c52315edfa833f8e67d9e3fa6c2a43a0ab2be0720ae06cbd60cc5 Jul 24 13:55:46.011293 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:46.011251 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" event={"ID":"51ef3fae-305b-4ed6-a06d-54471d352f74","Type":"ContainerStarted","Data":"7068f871226c52315edfa833f8e67d9e3fa6c2a43a0ab2be0720ae06cbd60cc5"} Jul 24 13:55:47.017828 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.017720 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" event={"ID":"72bb2246-854c-4c29-b01f-ac876a0d9514","Type":"ContainerStarted","Data":"6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34"} Jul 24 13:55:47.017828 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.017762 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" podUID="72bb2246-854c-4c29-b01f-ac876a0d9514" containerName="registry-server" containerID="cri-o://6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34" gracePeriod=2 Jul 24 13:55:47.019518 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.019487 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" event={"ID":"51ef3fae-305b-4ed6-a06d-54471d352f74","Type":"ContainerStarted","Data":"7015cb2a8548ac332fb53a09c7293ced0678dc103d7ffc65a229902516737e2c"} Jul 24 13:55:47.042306 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.042254 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" podStartSLOduration=1.176865944 podStartE2EDuration="3.042236916s" totalImagesPullingTime="1.865370972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:47.040742826 +0000 UTC m=+633.836085560" watchObservedRunningTime="2026-07-24 13:55:47.042236916 +0000 UTC m=+633.837579650" Jul 24 13:55:47.060254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.060195 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" podStartSLOduration=1.7949748749999999 podStartE2EDuration="3.060175406s" totalImagesPullingTime="1.265200531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:47.05895963 +0000 UTC m=+633.854302360" watchObservedRunningTime="2026-07-24 13:55:47.060175406 +0000 UTC m=+633.855518136" Jul 24 13:55:47.260421 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.260394 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:47.308961 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.308918 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wtwt\" (UniqueName: \"kubernetes.io/projected/72bb2246-854c-4c29-b01f-ac876a0d9514-kube-api-access-4wtwt\") pod \"72bb2246-854c-4c29-b01f-ac876a0d9514\" (UID: \"72bb2246-854c-4c29-b01f-ac876a0d9514\") " Jul 24 13:55:47.311231 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.311208 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72bb2246-854c-4c29-b01f-ac876a0d9514-kube-api-access-4wtwt" (OuterVolumeSpecName: "kube-api-access-4wtwt") pod "72bb2246-854c-4c29-b01f-ac876a0d9514" (UID: "72bb2246-854c-4c29-b01f-ac876a0d9514"). InnerVolumeSpecName "kube-api-access-4wtwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:55:47.409841 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:47.409790 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wtwt\" (UniqueName: \"kubernetes.io/projected/72bb2246-854c-4c29-b01f-ac876a0d9514-kube-api-access-4wtwt\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:55:48.024708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.024613 2555 generic.go:356] "Generic (PLEG): container finished" podID="72bb2246-854c-4c29-b01f-ac876a0d9514" containerID="6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34" exitCode=0 Jul 24 13:55:48.024708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.024659 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" Jul 24 13:55:48.024708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.024693 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" event={"ID":"72bb2246-854c-4c29-b01f-ac876a0d9514","Type":"ContainerDied","Data":"6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34"} Jul 24 13:55:48.025267 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.024749 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-f6vx4" event={"ID":"72bb2246-854c-4c29-b01f-ac876a0d9514","Type":"ContainerDied","Data":"b6282fd27c833d6286221b7473fd62a640ef3fb30e1abb2bdd8c6959ecda990e"} Jul 24 13:55:48.025267 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.024779 2555 scope.go:117] "RemoveContainer" containerID="6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34" Jul 24 13:55:48.033347 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.033327 2555 scope.go:117] "RemoveContainer" containerID="6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34" Jul 24 13:55:48.033641 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:55:48.033614 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34\": container with ID starting with 6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34 not found: ID does not exist" containerID="6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34" Jul 24 13:55:48.033738 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.033651 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34"} err="failed to get container status \"6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34\": rpc error: code = NotFound desc = could not find container \"6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34\": container with ID starting with 6c0555314f8c439ab1e956eb5976c826857ebf65e43869c9d41654cc7c3f6a34 not found: ID does not exist" Jul 24 13:55:48.040618 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.040589 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f6vx4"] Jul 24 13:55:48.044371 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:48.044346 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-f6vx4"] Jul 24 13:55:49.753046 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:49.753012 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72bb2246-854c-4c29-b01f-ac876a0d9514" path="/var/lib/kubelet/pods/72bb2246-854c-4c29-b01f-ac876a0d9514/volumes" Jul 24 13:55:55.277329 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:55.277283 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:55.277741 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:55.277357 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:55.299605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:55.299577 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:56.081305 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:56.081274 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-sztsl" Jul 24 13:55:56.479842 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:55:56.479768 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:12.997800 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:12.997764 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:16.640554 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:16.640518 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:17.741966 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.741934 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp"] Jul 24 13:56:17.742360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.742273 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72bb2246-854c-4c29-b01f-ac876a0d9514" containerName="registry-server" Jul 24 13:56:17.742360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.742285 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="72bb2246-854c-4c29-b01f-ac876a0d9514" containerName="registry-server" Jul 24 13:56:17.742438 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.742363 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="72bb2246-854c-4c29-b01f-ac876a0d9514" containerName="registry-server" Jul 24 13:56:17.744948 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.744921 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:17.747727 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.747700 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fxrj6\"" Jul 24 13:56:17.761341 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.761310 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp"] Jul 24 13:56:17.884953 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.884913 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcmlt\" (UniqueName: \"kubernetes.io/projected/316d1114-a2a6-46bf-95b6-521a3325d5bc-kube-api-access-bcmlt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:17.885155 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.885008 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/316d1114-a2a6-46bf-95b6-521a3325d5bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:17.986378 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.986336 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcmlt\" (UniqueName: \"kubernetes.io/projected/316d1114-a2a6-46bf-95b6-521a3325d5bc-kube-api-access-bcmlt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:17.986553 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.986400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/316d1114-a2a6-46bf-95b6-521a3325d5bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:17.986750 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.986734 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/316d1114-a2a6-46bf-95b6-521a3325d5bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:17.993899 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.993836 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:17.995635 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:17.995601 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcmlt\" (UniqueName: \"kubernetes.io/projected/316d1114-a2a6-46bf-95b6-521a3325d5bc-kube-api-access-bcmlt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:18.055813 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:18.055776 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:18.199397 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:18.199224 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp"] Jul 24 13:56:18.202408 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:56:18.202380 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod316d1114_a2a6_46bf_95b6_521a3325d5bc.slice/crio-d4498e3e84d491e01e4bb54923e78898c616923c1978f19ed1af21aff86b07b3 WatchSource:0}: Error finding container d4498e3e84d491e01e4bb54923e78898c616923c1978f19ed1af21aff86b07b3: Status 404 returned error can't find the container with id d4498e3e84d491e01e4bb54923e78898c616923c1978f19ed1af21aff86b07b3 Jul 24 13:56:18.393827 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:18.393794 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:19.165787 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:19.165753 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" event={"ID":"316d1114-a2a6-46bf-95b6-521a3325d5bc","Type":"ContainerStarted","Data":"d4498e3e84d491e01e4bb54923e78898c616923c1978f19ed1af21aff86b07b3"} Jul 24 13:56:19.423378 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:19.423290 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:21.005371 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:21.005327 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:22.049872 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:22.049836 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:23.416957 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:23.416846 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:24.191310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:24.191257 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" event={"ID":"316d1114-a2a6-46bf-95b6-521a3325d5bc","Type":"ContainerStarted","Data":"04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4"} Jul 24 13:56:24.191492 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:24.191342 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:24.229571 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:24.229513 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" podStartSLOduration=2.284462306 podStartE2EDuration="7.229495973s" totalImagesPullingTime="4.945033667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:24.227205939 +0000 UTC m=+671.022548669" watchObservedRunningTime="2026-07-24 13:56:24.229495973 +0000 UTC m=+671.024838704" Jul 24 13:56:24.635280 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:24.635246 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:25.108160 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.108126 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm"] Jul 24 13:56:25.111481 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.111464 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:25.114340 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.114315 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lcg62\"" Jul 24 13:56:25.114623 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.114605 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 24 13:56:25.120086 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.120061 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm"] Jul 24 13:56:25.144446 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.144409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phgm4\" (UniqueName: \"kubernetes.io/projected/f646fb79-0037-4ab6-84c2-fba0fa9c23fa-kube-api-access-phgm4\") pod \"dns-operator-controller-manager-864dd78c4c-qt5zm\" (UID: \"f646fb79-0037-4ab6-84c2-fba0fa9c23fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:25.245461 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.245423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phgm4\" (UniqueName: \"kubernetes.io/projected/f646fb79-0037-4ab6-84c2-fba0fa9c23fa-kube-api-access-phgm4\") pod \"dns-operator-controller-manager-864dd78c4c-qt5zm\" (UID: \"f646fb79-0037-4ab6-84c2-fba0fa9c23fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:25.256246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.256212 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phgm4\" (UniqueName: \"kubernetes.io/projected/f646fb79-0037-4ab6-84c2-fba0fa9c23fa-kube-api-access-phgm4\") pod \"dns-operator-controller-manager-864dd78c4c-qt5zm\" (UID: \"f646fb79-0037-4ab6-84c2-fba0fa9c23fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:25.424092 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.423969 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:25.570451 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:25.570423 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm"] Jul 24 13:56:25.572920 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:56:25.572894 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf646fb79_0037_4ab6_84c2_fba0fa9c23fa.slice/crio-466bef33010ecd685f7700d1236a2c79d9c0aad0f39b844aa25c14cf49a8b9e6 WatchSource:0}: Error finding container 466bef33010ecd685f7700d1236a2c79d9c0aad0f39b844aa25c14cf49a8b9e6: Status 404 returned error can't find the container with id 466bef33010ecd685f7700d1236a2c79d9c0aad0f39b844aa25c14cf49a8b9e6 Jul 24 13:56:26.035899 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.035862 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp"] Jul 24 13:56:26.040261 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.040236 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.043054 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.043021 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 24 13:56:26.043200 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.043171 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 24 13:56:26.043247 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.043206 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x5xxb\"" Jul 24 13:56:26.049100 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.049073 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:26.049767 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.049698 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp"] Jul 24 13:56:26.050440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.050414 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/261390f9-e358-4e1e-9ad7-91258b821347-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.050534 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.050464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9q72\" (UniqueName: \"kubernetes.io/projected/261390f9-e358-4e1e-9ad7-91258b821347-kube-api-access-q9q72\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.050598 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.050571 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/261390f9-e358-4e1e-9ad7-91258b821347-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.143648 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.143618 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:26.151112 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.151079 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9q72\" (UniqueName: \"kubernetes.io/projected/261390f9-e358-4e1e-9ad7-91258b821347-kube-api-access-q9q72\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.151279 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.151129 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/261390f9-e358-4e1e-9ad7-91258b821347-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.151279 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.151178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/261390f9-e358-4e1e-9ad7-91258b821347-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.151708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.151689 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/261390f9-e358-4e1e-9ad7-91258b821347-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.153806 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.153786 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/261390f9-e358-4e1e-9ad7-91258b821347-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.160279 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.160256 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9q72\" (UniqueName: \"kubernetes.io/projected/261390f9-e358-4e1e-9ad7-91258b821347-kube-api-access-q9q72\") pod \"kuadrant-console-plugin-66b5774985-jqccp\" (UID: \"261390f9-e358-4e1e-9ad7-91258b821347\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.202109 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.202072 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" event={"ID":"f646fb79-0037-4ab6-84c2-fba0fa9c23fa","Type":"ContainerStarted","Data":"466bef33010ecd685f7700d1236a2c79d9c0aad0f39b844aa25c14cf49a8b9e6"} Jul 24 13:56:26.351810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.351779 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" Jul 24 13:56:26.883228 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:26.883130 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp"] Jul 24 13:56:26.886326 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:56:26.886295 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod261390f9_e358_4e1e_9ad7_91258b821347.slice/crio-92d57884dad3163e0f3374d2db38b0a74d6c86fc83142a0bdef5a414bb38acad WatchSource:0}: Error finding container 92d57884dad3163e0f3374d2db38b0a74d6c86fc83142a0bdef5a414bb38acad: Status 404 returned error can't find the container with id 92d57884dad3163e0f3374d2db38b0a74d6c86fc83142a0bdef5a414bb38acad Jul 24 13:56:27.209825 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:27.209783 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" event={"ID":"261390f9-e358-4e1e-9ad7-91258b821347","Type":"ContainerStarted","Data":"92d57884dad3163e0f3374d2db38b0a74d6c86fc83142a0bdef5a414bb38acad"} Jul 24 13:56:28.218717 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:28.218679 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" event={"ID":"f646fb79-0037-4ab6-84c2-fba0fa9c23fa","Type":"ContainerStarted","Data":"c9f84b4a4a653605dbb3a5515862f778440408f9ea5f676c48434427d2e85c43"} Jul 24 13:56:28.219217 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:28.218826 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:28.236747 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:28.236695 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" podStartSLOduration=1.379448474 podStartE2EDuration="3.236673473s" totalImagesPullingTime="1.857224999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:28.23655406 +0000 UTC m=+675.031896816" watchObservedRunningTime="2026-07-24 13:56:28.236673473 +0000 UTC m=+675.032016204" Jul 24 13:56:28.397256 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:28.397221 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:28.468151 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:28.468074 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:35.199430 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:35.199397 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:37.010248 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.010218 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp"] Jul 24 13:56:37.010700 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.010441 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" containerName="manager" containerID="cri-o://04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4" gracePeriod=2 Jul 24 13:56:37.019480 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.019438 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp"] Jul 24 13:56:37.024807 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:37.024759 2555 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"316d1114-a2a6-46bf-95b6-521a3325d5bc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:37.033619 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.033588 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm"] Jul 24 13:56:37.034039 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.034020 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" containerName="manager" Jul 24 13:56:37.034144 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.034041 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" containerName="manager" Jul 24 13:56:37.034144 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.034141 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" containerName="manager" Jul 24 13:56:37.050327 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.048421 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm"] Jul 24 13:56:37.050327 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.048537 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.051353 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:37.051235 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:37.108909 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.108878 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g"] Jul 24 13:56:37.115643 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.115617 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.118332 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:37.118280 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:37.125506 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.125483 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g"] Jul 24 13:56:37.151741 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.151705 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpqnv\" (UniqueName: \"kubernetes.io/projected/46db023e-1565-489d-b91f-e4125c886702-kube-api-access-gpqnv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w69wm\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.151741 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.151747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmnvg\" (UniqueName: \"kubernetes.io/projected/13b1e7ea-a99e-4513-b6c2-43b59a261982-kube-api-access-nmnvg\") pod \"kuadrant-operator-controller-manager-86878c769f-jx44g\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.152004 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.151814 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46db023e-1565-489d-b91f-e4125c886702-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w69wm\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.152004 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.151886 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b1e7ea-a99e-4513-b6c2-43b59a261982-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jx44g\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.228578 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.228548 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:37.252975 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.252934 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b1e7ea-a99e-4513-b6c2-43b59a261982-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jx44g\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.253179 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.253032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpqnv\" (UniqueName: \"kubernetes.io/projected/46db023e-1565-489d-b91f-e4125c886702-kube-api-access-gpqnv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w69wm\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.253179 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.253066 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmnvg\" (UniqueName: \"kubernetes.io/projected/13b1e7ea-a99e-4513-b6c2-43b59a261982-kube-api-access-nmnvg\") pod \"kuadrant-operator-controller-manager-86878c769f-jx44g\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.253179 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.253140 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46db023e-1565-489d-b91f-e4125c886702-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w69wm\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.253407 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.253388 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b1e7ea-a99e-4513-b6c2-43b59a261982-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jx44g\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.253489 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.253470 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46db023e-1565-489d-b91f-e4125c886702-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w69wm\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.269925 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.269855 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmnvg\" (UniqueName: \"kubernetes.io/projected/13b1e7ea-a99e-4513-b6c2-43b59a261982-kube-api-access-nmnvg\") pod \"kuadrant-operator-controller-manager-86878c769f-jx44g\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:37.269925 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.269862 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpqnv\" (UniqueName: \"kubernetes.io/projected/46db023e-1565-489d-b91f-e4125c886702-kube-api-access-gpqnv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w69wm\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.326560 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.326528 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:37.416821 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.416779 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:37.429019 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:37.428963 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:39.227355 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:39.227311 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qt5zm" Jul 24 13:56:40.263330 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:40.263289 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:40.391461 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:40.391428 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:43.797837 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:43.797797 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:43.864429 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:43.864393 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:47.948892 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:47.948858 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:47.951404 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:47.951369 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:47.982472 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:47.982445 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g"] Jul 24 13:56:47.984197 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:56:47.984172 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b1e7ea_a99e_4513_b6c2_43b59a261982.slice/crio-f33c8a6dc9a356acf5cf1c1cfb5db2e015ab02240e1cdcdd97c3b2baa310b07c WatchSource:0}: Error finding container f33c8a6dc9a356acf5cf1c1cfb5db2e015ab02240e1cdcdd97c3b2baa310b07c: Status 404 returned error can't find the container with id f33c8a6dc9a356acf5cf1c1cfb5db2e015ab02240e1cdcdd97c3b2baa310b07c Jul 24 13:56:47.996734 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:47.996709 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm"] Jul 24 13:56:47.998814 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:56:47.998787 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46db023e_1565_489d_b91f_e4125c886702.slice/crio-b7bc2f5c59637376c9479a3c091b9362d6722bd7385342ae8cb57774e7ead079 WatchSource:0}: Error finding container b7bc2f5c59637376c9479a3c091b9362d6722bd7385342ae8cb57774e7ead079: Status 404 returned error can't find the container with id b7bc2f5c59637376c9479a3c091b9362d6722bd7385342ae8cb57774e7ead079 Jul 24 13:56:48.054337 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.054311 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcmlt\" (UniqueName: \"kubernetes.io/projected/316d1114-a2a6-46bf-95b6-521a3325d5bc-kube-api-access-bcmlt\") pod \"316d1114-a2a6-46bf-95b6-521a3325d5bc\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " Jul 24 13:56:48.054519 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.054345 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/316d1114-a2a6-46bf-95b6-521a3325d5bc-extensions-socket-volume\") pod \"316d1114-a2a6-46bf-95b6-521a3325d5bc\" (UID: \"316d1114-a2a6-46bf-95b6-521a3325d5bc\") " Jul 24 13:56:48.054883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.054862 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/316d1114-a2a6-46bf-95b6-521a3325d5bc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "316d1114-a2a6-46bf-95b6-521a3325d5bc" (UID: "316d1114-a2a6-46bf-95b6-521a3325d5bc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:56:48.056445 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.056418 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/316d1114-a2a6-46bf-95b6-521a3325d5bc-kube-api-access-bcmlt" (OuterVolumeSpecName: "kube-api-access-bcmlt") pod "316d1114-a2a6-46bf-95b6-521a3325d5bc" (UID: "316d1114-a2a6-46bf-95b6-521a3325d5bc"). InnerVolumeSpecName "kube-api-access-bcmlt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:56:48.155492 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.155438 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcmlt\" (UniqueName: \"kubernetes.io/projected/316d1114-a2a6-46bf-95b6-521a3325d5bc-kube-api-access-bcmlt\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:56:48.155492 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.155475 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/316d1114-a2a6-46bf-95b6-521a3325d5bc-extensions-socket-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:56:48.328279 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.328240 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" event={"ID":"261390f9-e358-4e1e-9ad7-91258b821347","Type":"ContainerStarted","Data":"d69cb5ef418f1c3490d942862317dfdce4f91852709ec05750de1aacda76343b"} Jul 24 13:56:48.330024 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.329976 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" event={"ID":"46db023e-1565-489d-b91f-e4125c886702","Type":"ContainerStarted","Data":"4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0"} Jul 24 13:56:48.330181 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.330052 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:48.330181 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.330069 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" event={"ID":"46db023e-1565-489d-b91f-e4125c886702","Type":"ContainerStarted","Data":"b7bc2f5c59637376c9479a3c091b9362d6722bd7385342ae8cb57774e7ead079"} Jul 24 13:56:48.330753 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:48.330730 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:48.331215 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.331195 2555 generic.go:356] "Generic (PLEG): container finished" podID="316d1114-a2a6-46bf-95b6-521a3325d5bc" containerID="04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4" exitCode=0 Jul 24 13:56:48.331280 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.331231 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:48.331280 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.331267 2555 scope.go:117] "RemoveContainer" containerID="04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4" Jul 24 13:56:48.332803 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.332781 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" event={"ID":"13b1e7ea-a99e-4513-b6c2-43b59a261982","Type":"ContainerStarted","Data":"d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7"} Jul 24 13:56:48.332903 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.332825 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" event={"ID":"13b1e7ea-a99e-4513-b6c2-43b59a261982","Type":"ContainerStarted","Data":"f33c8a6dc9a356acf5cf1c1cfb5db2e015ab02240e1cdcdd97c3b2baa310b07c"} Jul 24 13:56:48.332903 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.332861 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:48.341246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.341223 2555 scope.go:117] "RemoveContainer" containerID="04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4" Jul 24 13:56:48.341626 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:48.341597 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4\": container with ID starting with 04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4 not found: ID does not exist" containerID="04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4" Jul 24 13:56:48.341703 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.341641 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4"} err="failed to get container status \"04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4\": rpc error: code = NotFound desc = could not find container \"04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4\": container with ID starting with 04ccc551dc14dca58a2e0a00d34c25cccbb6cc987fee0e1bb8f20241dd1eb0f4 not found: ID does not exist" Jul 24 13:56:48.348917 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.348865 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-jqccp" podStartSLOduration=1.346263364 podStartE2EDuration="22.34884953s" totalImagesPullingTime="21.002586166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:48.348049533 +0000 UTC m=+695.143392264" watchObservedRunningTime="2026-07-24 13:56:48.34884953 +0000 UTC m=+695.144192260" Jul 24 13:56:48.350072 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:56:48.350046 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9j9pp\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9j9pp" Jul 24 13:56:48.390452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.390404 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" podStartSLOduration=11.390386088 podStartE2EDuration="11.390386088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:48.369802703 +0000 UTC m=+695.165145444" watchObservedRunningTime="2026-07-24 13:56:48.390386088 +0000 UTC m=+695.185728818" Jul 24 13:56:48.391865 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:48.391826 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" podStartSLOduration=11.391811403 podStartE2EDuration="11.391811403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:48.38946258 +0000 UTC m=+695.184805311" watchObservedRunningTime="2026-07-24 13:56:48.391811403 +0000 UTC m=+695.187154132" Jul 24 13:56:49.665304 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:49.665270 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:49.749327 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:49.749294 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="316d1114-a2a6-46bf-95b6-521a3325d5bc" path="/var/lib/kubelet/pods/316d1114-a2a6-46bf-95b6-521a3325d5bc/volumes" Jul 24 13:56:49.765345 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:49.765314 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:55.419810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.419770 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58bc85ccc6-7rx6k"] Jul 24 13:56:55.483261 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.483226 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:55.565915 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.565879 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58bc85ccc6-7rx6k"] Jul 24 13:56:55.566106 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.566021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.613646 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.613612 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:55.725434 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnfxm\" (UniqueName: \"kubernetes.io/projected/4d68954f-8c36-4438-8ff2-4d73f5274f4b-kube-api-access-tnfxm\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.725434 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725393 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-oauth-config\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.725434 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725412 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-oauth-serving-cert\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.725667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725515 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-serving-cert\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.725667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725555 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-config\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.725667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725571 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-trusted-ca-bundle\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.725667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.725597 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-service-ca\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826158 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826120 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-serving-cert\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-config\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826210 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-trusted-ca-bundle\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826351 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826250 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-service-ca\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826549 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826436 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnfxm\" (UniqueName: \"kubernetes.io/projected/4d68954f-8c36-4438-8ff2-4d73f5274f4b-kube-api-access-tnfxm\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826549 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826516 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-oauth-config\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826549 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826543 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-oauth-serving-cert\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.826972 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.826954 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-config\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.827212 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.827191 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-service-ca\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.827310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.827288 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-oauth-serving-cert\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.827468 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.827447 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d68954f-8c36-4438-8ff2-4d73f5274f4b-trusted-ca-bundle\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.828897 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.828876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-serving-cert\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.829048 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.829029 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d68954f-8c36-4438-8ff2-4d73f5274f4b-console-oauth-config\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.835005 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.834965 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnfxm\" (UniqueName: \"kubernetes.io/projected/4d68954f-8c36-4438-8ff2-4d73f5274f4b-kube-api-access-tnfxm\") pod \"console-58bc85ccc6-7rx6k\" (UID: \"4d68954f-8c36-4438-8ff2-4d73f5274f4b\") " pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:55.875715 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:55.875677 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:56:56.040857 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:56.040831 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58bc85ccc6-7rx6k"] Jul 24 13:56:56.043743 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:56:56.043714 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d68954f_8c36_4438_8ff2_4d73f5274f4b.slice/crio-f019e2e08ec9db10311e21aa40795674bf65a4882ca7da2cd8827b6650bf75ca WatchSource:0}: Error finding container f019e2e08ec9db10311e21aa40795674bf65a4882ca7da2cd8827b6650bf75ca: Status 404 returned error can't find the container with id f019e2e08ec9db10311e21aa40795674bf65a4882ca7da2cd8827b6650bf75ca Jul 24 13:56:56.220542 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:56.220510 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:56.320940 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:56.320909 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:56:56.374682 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:56.374649 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58bc85ccc6-7rx6k" event={"ID":"4d68954f-8c36-4438-8ff2-4d73f5274f4b","Type":"ContainerStarted","Data":"28d03d75ac80609937ce6e0890b990e0c2133c56e4766f0e2c436fad0163beda"} Jul 24 13:56:56.374820 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:56.374701 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58bc85ccc6-7rx6k" event={"ID":"4d68954f-8c36-4438-8ff2-4d73f5274f4b","Type":"ContainerStarted","Data":"f019e2e08ec9db10311e21aa40795674bf65a4882ca7da2cd8827b6650bf75ca"} Jul 24 13:56:56.392875 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:56.392826 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58bc85ccc6-7rx6k" podStartSLOduration=1.392809692 podStartE2EDuration="1.392809692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:56.392332149 +0000 UTC m=+703.187674872" watchObservedRunningTime="2026-07-24 13:56:56.392809692 +0000 UTC m=+703.188152421" Jul 24 13:56:59.341801 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.341764 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 13:56:59.342197 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.341971 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:59.417272 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.417239 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm"] Jul 24 13:56:59.417440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.417419 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" podUID="46db023e-1565-489d-b91f-e4125c886702" containerName="manager" containerID="cri-o://4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0" gracePeriod=10 Jul 24 13:56:59.686611 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.686582 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:56:59.763162 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.763131 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpqnv\" (UniqueName: \"kubernetes.io/projected/46db023e-1565-489d-b91f-e4125c886702-kube-api-access-gpqnv\") pod \"46db023e-1565-489d-b91f-e4125c886702\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " Jul 24 13:56:59.763360 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.763219 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46db023e-1565-489d-b91f-e4125c886702-extensions-socket-volume\") pod \"46db023e-1565-489d-b91f-e4125c886702\" (UID: \"46db023e-1565-489d-b91f-e4125c886702\") " Jul 24 13:56:59.763580 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.763549 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46db023e-1565-489d-b91f-e4125c886702-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "46db023e-1565-489d-b91f-e4125c886702" (UID: "46db023e-1565-489d-b91f-e4125c886702"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:56:59.765661 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.765633 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46db023e-1565-489d-b91f-e4125c886702-kube-api-access-gpqnv" (OuterVolumeSpecName: "kube-api-access-gpqnv") pod "46db023e-1565-489d-b91f-e4125c886702" (UID: "46db023e-1565-489d-b91f-e4125c886702"). InnerVolumeSpecName "kube-api-access-gpqnv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:56:59.864290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.864205 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/46db023e-1565-489d-b91f-e4125c886702-extensions-socket-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:56:59.864290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:56:59.864246 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpqnv\" (UniqueName: \"kubernetes.io/projected/46db023e-1565-489d-b91f-e4125c886702-kube-api-access-gpqnv\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:00.263054 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.262939 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:00.364035 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.364003 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:00.395714 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.395678 2555 generic.go:356] "Generic (PLEG): container finished" podID="46db023e-1565-489d-b91f-e4125c886702" containerID="4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0" exitCode=0 Jul 24 13:57:00.395714 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.395720 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" Jul 24 13:57:00.395914 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.395770 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" event={"ID":"46db023e-1565-489d-b91f-e4125c886702","Type":"ContainerDied","Data":"4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0"} Jul 24 13:57:00.395914 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.395816 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm" event={"ID":"46db023e-1565-489d-b91f-e4125c886702","Type":"ContainerDied","Data":"b7bc2f5c59637376c9479a3c091b9362d6722bd7385342ae8cb57774e7ead079"} Jul 24 13:57:00.395914 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.395840 2555 scope.go:117] "RemoveContainer" containerID="4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0" Jul 24 13:57:00.405457 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.405441 2555 scope.go:117] "RemoveContainer" containerID="4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0" Jul 24 13:57:00.405762 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:57:00.405743 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0\": container with ID starting with 4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0 not found: ID does not exist" containerID="4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0" Jul 24 13:57:00.405794 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.405772 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0"} err="failed to get container status \"4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0\": rpc error: code = NotFound desc = could not find container \"4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0\": container with ID starting with 4f8a65c28459ef3f6c8b9bb964d22603b95c15d48a8cd2e8ba51406f8472aaa0 not found: ID does not exist" Jul 24 13:57:00.418230 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.418197 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm"] Jul 24 13:57:00.430432 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:00.430404 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w69wm"] Jul 24 13:57:01.456597 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:01.456564 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:01.557637 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:01.557603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:01.749274 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:01.749202 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46db023e-1565-489d-b91f-e4125c886702" path="/var/lib/kubelet/pods/46db023e-1565-489d-b91f-e4125c886702/volumes" Jul 24 13:57:03.268027 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:03.267972 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:03.368630 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:03.368587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:05.732393 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:05.732281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:05.876141 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:05.876099 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:57:05.876317 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:05.876167 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:57:05.881175 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:05.881148 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:57:05.912963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:05.912910 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:06.428662 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:06.428633 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58bc85ccc6-7rx6k" Jul 24 13:57:06.476907 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:06.476872 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bfccfc68f-8ndxv"] Jul 24 13:57:06.590268 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:06.590240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:06.723027 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:06.722911 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:06.790082 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:06.790033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:15.730746 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.730713 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn"] Jul 24 13:57:15.731155 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.731098 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46db023e-1565-489d-b91f-e4125c886702" containerName="manager" Jul 24 13:57:15.731155 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.731110 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="46db023e-1565-489d-b91f-e4125c886702" containerName="manager" Jul 24 13:57:15.731238 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.731158 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="46db023e-1565-489d-b91f-e4125c886702" containerName="manager" Jul 24 13:57:15.733969 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.733946 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.736779 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.736751 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qcjcf\"" Jul 24 13:57:15.750964 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.750934 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn"] Jul 24 13:57:15.750964 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.750947 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:15.804551 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804512 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.804759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804577 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.804759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804610 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.804759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804646 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.804759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804667 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.804759 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804714 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.805073 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804782 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd838d0a-45bf-4e91-8473-41bb74497fc1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.805073 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804818 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mxrp\" (UniqueName: \"kubernetes.io/projected/dd838d0a-45bf-4e91-8473-41bb74497fc1-kube-api-access-7mxrp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.805073 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.804851 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.902907 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.902871 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:15.905257 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905228 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905414 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905267 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905414 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905304 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905414 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905414 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905362 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905414 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905403 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd838d0a-45bf-4e91-8473-41bb74497fc1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905671 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905431 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mxrp\" (UniqueName: \"kubernetes.io/projected/dd838d0a-45bf-4e91-8473-41bb74497fc1-kube-api-access-7mxrp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905671 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905466 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905671 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905516 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905822 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905665 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905822 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905694 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.905930 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.905842 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.906138 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.906068 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.906381 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.906262 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd838d0a-45bf-4e91-8473-41bb74497fc1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.907881 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.907861 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.908004 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.907935 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.913719 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.913682 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mxrp\" (UniqueName: \"kubernetes.io/projected/dd838d0a-45bf-4e91-8473-41bb74497fc1-kube-api-access-7mxrp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:15.913820 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:15.913721 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd838d0a-45bf-4e91-8473-41bb74497fc1-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-qkcrn\" (UID: \"dd838d0a-45bf-4e91-8473-41bb74497fc1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:16.045653 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.045617 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:16.190934 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.190908 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn"] Jul 24 13:57:16.193075 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:57:16.193048 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd838d0a_45bf_4e91_8473_41bb74497fc1.slice/crio-714d835e9cdf35536b4878068798082c96b7f78f74501242abc22ea209efe835 WatchSource:0}: Error finding container 714d835e9cdf35536b4878068798082c96b7f78f74501242abc22ea209efe835: Status 404 returned error can't find the container with id 714d835e9cdf35536b4878068798082c96b7f78f74501242abc22ea209efe835 Jul 24 13:57:16.195329 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.195289 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:57:16.195440 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.195377 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:57:16.195502 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.195441 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:57:16.471458 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.471369 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" event={"ID":"dd838d0a-45bf-4e91-8473-41bb74497fc1","Type":"ContainerStarted","Data":"97109490e34f4706a4d5ecde8cefc3986ea33365f5fe6299939f193bf75a59f8"} Jul 24 13:57:16.471458 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.471430 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" event={"ID":"dd838d0a-45bf-4e91-8473-41bb74497fc1","Type":"ContainerStarted","Data":"714d835e9cdf35536b4878068798082c96b7f78f74501242abc22ea209efe835"} Jul 24 13:57:16.492792 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:16.492723 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" podStartSLOduration=1.492706586 podStartE2EDuration="1.492706586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:57:16.490823319 +0000 UTC m=+723.286166050" watchObservedRunningTime="2026-07-24 13:57:16.492706586 +0000 UTC m=+723.288049315" Jul 24 13:57:17.046144 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.046104 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:17.051336 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.051309 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:17.450080 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.449968 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:17.476739 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.476709 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:17.477853 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.477826 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:57:17.572932 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.572903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:17.741102 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:17.741010 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:18.300866 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.300832 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-tfhns"] Jul 24 13:57:18.313098 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.313071 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfhns"] Jul 24 13:57:18.313276 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.313179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:18.315853 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.315828 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cdggs\"" Jul 24 13:57:18.328821 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.328785 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhbj\" (UniqueName: \"kubernetes.io/projected/e1efda15-764f-4893-b088-ddcea65640c9-kube-api-access-lfhbj\") pod \"authorino-58cd955675-tfhns\" (UID: \"e1efda15-764f-4893-b088-ddcea65640c9\") " pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:18.336306 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.336273 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:18.413863 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.413827 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:18.429963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.429922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhbj\" (UniqueName: \"kubernetes.io/projected/e1efda15-764f-4893-b088-ddcea65640c9-kube-api-access-lfhbj\") pod \"authorino-58cd955675-tfhns\" (UID: \"e1efda15-764f-4893-b088-ddcea65640c9\") " pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:18.442467 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.442436 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhbj\" (UniqueName: \"kubernetes.io/projected/e1efda15-764f-4893-b088-ddcea65640c9-kube-api-access-lfhbj\") pod \"authorino-58cd955675-tfhns\" (UID: \"e1efda15-764f-4893-b088-ddcea65640c9\") " pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:18.476178 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.476137 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mztmg"] Jul 24 13:57:18.479710 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.479687 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:18.487843 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.487815 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mztmg"] Jul 24 13:57:18.530541 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.530503 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjg2\" (UniqueName: \"kubernetes.io/projected/4b66006d-d0a6-485c-9f82-343b1d11f776-kube-api-access-8mjg2\") pod \"authorino-75d556bd77-mztmg\" (UID: \"4b66006d-d0a6-485c-9f82-343b1d11f776\") " pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:18.623708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.623616 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:18.632189 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.632164 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjg2\" (UniqueName: \"kubernetes.io/projected/4b66006d-d0a6-485c-9f82-343b1d11f776-kube-api-access-8mjg2\") pod \"authorino-75d556bd77-mztmg\" (UID: \"4b66006d-d0a6-485c-9f82-343b1d11f776\") " pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:18.643593 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.643563 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjg2\" (UniqueName: \"kubernetes.io/projected/4b66006d-d0a6-485c-9f82-343b1d11f776-kube-api-access-8mjg2\") pod \"authorino-75d556bd77-mztmg\" (UID: \"4b66006d-d0a6-485c-9f82-343b1d11f776\") " pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:18.765371 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.765316 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfhns"] Jul 24 13:57:18.767666 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:57:18.767631 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1efda15_764f_4893_b088_ddcea65640c9.slice/crio-486a1a6dbaa223caf7bc5207eb19cbc1b66d53bc3cef5eec69619e5136956b80 WatchSource:0}: Error finding container 486a1a6dbaa223caf7bc5207eb19cbc1b66d53bc3cef5eec69619e5136956b80: Status 404 returned error can't find the container with id 486a1a6dbaa223caf7bc5207eb19cbc1b66d53bc3cef5eec69619e5136956b80 Jul 24 13:57:18.790452 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.790412 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:18.928514 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:18.928474 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mztmg"] Jul 24 13:57:18.930899 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:57:18.930874 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b66006d_d0a6_485c_9f82_343b1d11f776.slice/crio-9b3860f2171831bd8fef086ba5bcc9ffff7edd41a0147ca4ca371b530f6ccf90 WatchSource:0}: Error finding container 9b3860f2171831bd8fef086ba5bcc9ffff7edd41a0147ca4ca371b530f6ccf90: Status 404 returned error can't find the container with id 9b3860f2171831bd8fef086ba5bcc9ffff7edd41a0147ca4ca371b530f6ccf90 Jul 24 13:57:19.492881 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:19.492841 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfhns" event={"ID":"e1efda15-764f-4893-b088-ddcea65640c9","Type":"ContainerStarted","Data":"486a1a6dbaa223caf7bc5207eb19cbc1b66d53bc3cef5eec69619e5136956b80"} Jul 24 13:57:19.500044 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:19.500004 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mztmg" event={"ID":"4b66006d-d0a6-485c-9f82-343b1d11f776","Type":"ContainerStarted","Data":"9b3860f2171831bd8fef086ba5bcc9ffff7edd41a0147ca4ca371b530f6ccf90"} Jul 24 13:57:22.518441 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:22.518408 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mztmg" event={"ID":"4b66006d-d0a6-485c-9f82-343b1d11f776","Type":"ContainerStarted","Data":"c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117"} Jul 24 13:57:22.519870 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:22.519848 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfhns" event={"ID":"e1efda15-764f-4893-b088-ddcea65640c9","Type":"ContainerStarted","Data":"e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4"} Jul 24 13:57:22.540671 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:22.540615 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mztmg" podStartSLOduration=1.162861324 podStartE2EDuration="4.540598274s" totalImagesPullingTime="3.37773695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:57:22.538807062 +0000 UTC m=+729.334149791" watchObservedRunningTime="2026-07-24 13:57:22.540598274 +0000 UTC m=+729.335941003" Jul 24 13:57:22.559015 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:22.558883 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-tfhns" podStartSLOduration=1.007308187 podStartE2EDuration="4.55886446s" totalImagesPullingTime="3.551556273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:57:22.557530304 +0000 UTC m=+729.352873035" watchObservedRunningTime="2026-07-24 13:57:22.55886446 +0000 UTC m=+729.354207194" Jul 24 13:57:22.583217 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:22.583183 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfhns"] Jul 24 13:57:24.528814 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:24.528772 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-tfhns" podUID="e1efda15-764f-4893-b088-ddcea65640c9" containerName="authorino" containerID="cri-o://e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4" gracePeriod=30 Jul 24 13:57:24.796429 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:24.796403 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:24.893667 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:24.893618 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfhbj\" (UniqueName: \"kubernetes.io/projected/e1efda15-764f-4893-b088-ddcea65640c9-kube-api-access-lfhbj\") pod \"e1efda15-764f-4893-b088-ddcea65640c9\" (UID: \"e1efda15-764f-4893-b088-ddcea65640c9\") " Jul 24 13:57:24.895871 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:24.895842 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1efda15-764f-4893-b088-ddcea65640c9-kube-api-access-lfhbj" (OuterVolumeSpecName: "kube-api-access-lfhbj") pod "e1efda15-764f-4893-b088-ddcea65640c9" (UID: "e1efda15-764f-4893-b088-ddcea65640c9"). InnerVolumeSpecName "kube-api-access-lfhbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:24.995171 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:24.995123 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfhbj\" (UniqueName: \"kubernetes.io/projected/e1efda15-764f-4893-b088-ddcea65640c9-kube-api-access-lfhbj\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:25.534410 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.534374 2555 generic.go:356] "Generic (PLEG): container finished" podID="e1efda15-764f-4893-b088-ddcea65640c9" containerID="e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4" exitCode=0 Jul 24 13:57:25.534410 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.534418 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-tfhns" Jul 24 13:57:25.534963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.534458 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfhns" event={"ID":"e1efda15-764f-4893-b088-ddcea65640c9","Type":"ContainerDied","Data":"e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4"} Jul 24 13:57:25.534963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.534500 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-tfhns" event={"ID":"e1efda15-764f-4893-b088-ddcea65640c9","Type":"ContainerDied","Data":"486a1a6dbaa223caf7bc5207eb19cbc1b66d53bc3cef5eec69619e5136956b80"} Jul 24 13:57:25.534963 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.534519 2555 scope.go:117] "RemoveContainer" containerID="e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4" Jul 24 13:57:25.543412 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.543395 2555 scope.go:117] "RemoveContainer" containerID="e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4" Jul 24 13:57:25.543689 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:57:25.543672 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4\": container with ID starting with e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4 not found: ID does not exist" containerID="e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4" Jul 24 13:57:25.543727 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.543701 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4"} err="failed to get container status \"e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4\": rpc error: code = NotFound desc = could not find container \"e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4\": container with ID starting with e5b6258137f5042d1241564cc31b91b93ff3f73b26fe9aad1afad09c8b7904a4 not found: ID does not exist" Jul 24 13:57:25.560831 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.560795 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfhns"] Jul 24 13:57:25.566522 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.566492 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-tfhns"] Jul 24 13:57:25.749807 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:25.749772 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1efda15-764f-4893-b088-ddcea65640c9" path="/var/lib/kubelet/pods/e1efda15-764f-4893-b088-ddcea65640c9/volumes" Jul 24 13:57:31.497652 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.497611 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bfccfc68f-8ndxv" podUID="0422a600-5b0e-44be-a9ef-aaf3b6a379c6" containerName="console" containerID="cri-o://de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e" gracePeriod=15 Jul 24 13:57:31.741678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.741644 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:57:31.857998 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.857953 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-trusted-ca-bundle\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858058 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-config\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858082 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-serving-cert\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858132 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqhtg\" (UniqueName: \"kubernetes.io/projected/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-kube-api-access-kqhtg\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858328 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858183 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-oauth-config\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858328 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858215 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-service-ca\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858328 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858239 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-oauth-serving-cert\") pod \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\" (UID: \"0422a600-5b0e-44be-a9ef-aaf3b6a379c6\") " Jul 24 13:57:31.858473 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858424 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:31.858554 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858522 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-config" (OuterVolumeSpecName: "console-config") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:31.858758 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858734 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-trusted-ca-bundle\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:31.858758 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858759 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:31.859142 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.858771 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:31.859142 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.859110 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:31.860554 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.860526 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:57:31.860668 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.860560 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:57:31.860668 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.860576 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-kube-api-access-kqhtg" (OuterVolumeSpecName: "kube-api-access-kqhtg") pod "0422a600-5b0e-44be-a9ef-aaf3b6a379c6" (UID: "0422a600-5b0e-44be-a9ef-aaf3b6a379c6"). InnerVolumeSpecName "kube-api-access-kqhtg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:31.960099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.960064 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:31.960099 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.960100 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqhtg\" (UniqueName: \"kubernetes.io/projected/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-kube-api-access-kqhtg\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:31.960299 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.960111 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-console-oauth-config\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:31.960299 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.960122 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-service-ca\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:31.960299 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:31.960131 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0422a600-5b0e-44be-a9ef-aaf3b6a379c6-oauth-serving-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:32.573881 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.573843 2555 generic.go:356] "Generic (PLEG): container finished" podID="0422a600-5b0e-44be-a9ef-aaf3b6a379c6" containerID="de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e" exitCode=0 Jul 24 13:57:32.573881 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.573887 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bfccfc68f-8ndxv" Jul 24 13:57:32.574400 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.573928 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfccfc68f-8ndxv" event={"ID":"0422a600-5b0e-44be-a9ef-aaf3b6a379c6","Type":"ContainerDied","Data":"de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e"} Jul 24 13:57:32.574400 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.573999 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bfccfc68f-8ndxv" event={"ID":"0422a600-5b0e-44be-a9ef-aaf3b6a379c6","Type":"ContainerDied","Data":"4c6af0202a3803bd0e123b9881d08a89c445dc52d120bf0a749438071ff01350"} Jul 24 13:57:32.574400 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.574013 2555 scope.go:117] "RemoveContainer" containerID="de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e" Jul 24 13:57:32.583246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.583229 2555 scope.go:117] "RemoveContainer" containerID="de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e" Jul 24 13:57:32.583526 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:57:32.583507 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e\": container with ID starting with de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e not found: ID does not exist" containerID="de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e" Jul 24 13:57:32.583582 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.583537 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e"} err="failed to get container status \"de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e\": rpc error: code = NotFound desc = could not find container \"de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e\": container with ID starting with de00be3e76cdd1d21fc6edacfde958aed33f91b2e175e9532a041cdc318c635e not found: ID does not exist" Jul 24 13:57:32.596608 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.596574 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bfccfc68f-8ndxv"] Jul 24 13:57:32.600429 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:32.600407 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bfccfc68f-8ndxv"] Jul 24 13:57:33.632550 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:33.632522 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:33.737087 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:33.737056 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:33.749971 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:33.749936 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0422a600-5b0e-44be-a9ef-aaf3b6a379c6" path="/var/lib/kubelet/pods/0422a600-5b0e-44be-a9ef-aaf3b6a379c6/volumes" Jul 24 13:57:34.128002 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:34.127922 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:46.359701 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.359667 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:46.482162 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.482129 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:46.587837 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.587793 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q5vwm"] Jul 24 13:57:46.588254 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.588235 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1efda15-764f-4893-b088-ddcea65640c9" containerName="authorino" Jul 24 13:57:46.588349 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.588256 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1efda15-764f-4893-b088-ddcea65640c9" containerName="authorino" Jul 24 13:57:46.588349 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.588277 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0422a600-5b0e-44be-a9ef-aaf3b6a379c6" containerName="console" Jul 24 13:57:46.588349 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.588286 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0422a600-5b0e-44be-a9ef-aaf3b6a379c6" containerName="console" Jul 24 13:57:46.588493 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.588378 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1efda15-764f-4893-b088-ddcea65640c9" containerName="authorino" Jul 24 13:57:46.588493 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.588392 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0422a600-5b0e-44be-a9ef-aaf3b6a379c6" containerName="console" Jul 24 13:57:46.591204 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.591179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:46.598570 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.598550 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q5vwm"] Jul 24 13:57:46.701802 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.701711 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx57c\" (UniqueName: \"kubernetes.io/projected/a93d182c-6795-4603-9b70-d5ad8b30cd98-kube-api-access-tx57c\") pod \"authorino-5d4b67c446-q5vwm\" (UID: \"a93d182c-6795-4603-9b70-d5ad8b30cd98\") " pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:46.768639 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.768602 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q5vwm"] Jul 24 13:57:46.768893 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:57:46.768873 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-tx57c], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-q5vwm" podUID="a93d182c-6795-4603-9b70-d5ad8b30cd98" Jul 24 13:57:46.802442 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.802400 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx57c\" (UniqueName: \"kubernetes.io/projected/a93d182c-6795-4603-9b70-d5ad8b30cd98-kube-api-access-tx57c\") pod \"authorino-5d4b67c446-q5vwm\" (UID: \"a93d182c-6795-4603-9b70-d5ad8b30cd98\") " pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:46.816032 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.816002 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx57c\" (UniqueName: \"kubernetes.io/projected/a93d182c-6795-4603-9b70-d5ad8b30cd98-kube-api-access-tx57c\") pod \"authorino-5d4b67c446-q5vwm\" (UID: \"a93d182c-6795-4603-9b70-d5ad8b30cd98\") " pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:46.877933 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.877896 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c4f568fc6-wd884"] Jul 24 13:57:46.880837 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.880817 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:46.883629 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.883608 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 13:57:46.891808 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:46.891780 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c4f568fc6-wd884"] Jul 24 13:57:47.004697 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.004611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkjlw\" (UniqueName: \"kubernetes.io/projected/55864e0e-4647-465e-8f7f-dd1ee178e170-kube-api-access-gkjlw\") pod \"authorino-6c4f568fc6-wd884\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.004697 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.004656 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55864e0e-4647-465e-8f7f-dd1ee178e170-tls-cert\") pod \"authorino-6c4f568fc6-wd884\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.105627 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.105597 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkjlw\" (UniqueName: \"kubernetes.io/projected/55864e0e-4647-465e-8f7f-dd1ee178e170-kube-api-access-gkjlw\") pod \"authorino-6c4f568fc6-wd884\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.105742 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.105638 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55864e0e-4647-465e-8f7f-dd1ee178e170-tls-cert\") pod \"authorino-6c4f568fc6-wd884\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.108484 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.108452 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55864e0e-4647-465e-8f7f-dd1ee178e170-tls-cert\") pod \"authorino-6c4f568fc6-wd884\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.114642 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.114614 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkjlw\" (UniqueName: \"kubernetes.io/projected/55864e0e-4647-465e-8f7f-dd1ee178e170-kube-api-access-gkjlw\") pod \"authorino-6c4f568fc6-wd884\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.190820 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.190781 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 13:57:47.331310 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.331278 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c4f568fc6-wd884"] Jul 24 13:57:47.642388 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.642350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4f568fc6-wd884" event={"ID":"55864e0e-4647-465e-8f7f-dd1ee178e170","Type":"ContainerStarted","Data":"ca08e90b318726736bcda2d2e5f8acaf50cae0f4730293e67ed2fe42e38c6ce3"} Jul 24 13:57:47.642756 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.642420 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:47.649495 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.649473 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:47.813781 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.813751 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx57c\" (UniqueName: \"kubernetes.io/projected/a93d182c-6795-4603-9b70-d5ad8b30cd98-kube-api-access-tx57c\") pod \"a93d182c-6795-4603-9b70-d5ad8b30cd98\" (UID: \"a93d182c-6795-4603-9b70-d5ad8b30cd98\") " Jul 24 13:57:47.816061 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.816035 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a93d182c-6795-4603-9b70-d5ad8b30cd98-kube-api-access-tx57c" (OuterVolumeSpecName: "kube-api-access-tx57c") pod "a93d182c-6795-4603-9b70-d5ad8b30cd98" (UID: "a93d182c-6795-4603-9b70-d5ad8b30cd98"). InnerVolumeSpecName "kube-api-access-tx57c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:47.915453 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:47.915367 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx57c\" (UniqueName: \"kubernetes.io/projected/a93d182c-6795-4603-9b70-d5ad8b30cd98-kube-api-access-tx57c\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:48.648411 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.648380 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-q5vwm" Jul 24 13:57:48.648824 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.648427 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4f568fc6-wd884" event={"ID":"55864e0e-4647-465e-8f7f-dd1ee178e170","Type":"ContainerStarted","Data":"afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e"} Jul 24 13:57:48.665766 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.665716 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c4f568fc6-wd884" podStartSLOduration=2.358987322 podStartE2EDuration="2.665700324s" totalImagesPullingTime="306.713002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:57:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:57:48.665211155 +0000 UTC m=+755.460553885" watchObservedRunningTime="2026-07-24 13:57:48.665700324 +0000 UTC m=+755.461043100" Jul 24 13:57:48.707592 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.707548 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mztmg"] Jul 24 13:57:48.707840 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.707798 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mztmg" podUID="4b66006d-d0a6-485c-9f82-343b1d11f776" containerName="authorino" containerID="cri-o://c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117" gracePeriod=30 Jul 24 13:57:48.711246 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.711209 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q5vwm"] Jul 24 13:57:48.720627 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.720599 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-q5vwm"] Jul 24 13:57:48.967126 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:48.967101 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:49.125718 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.125681 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mjg2\" (UniqueName: \"kubernetes.io/projected/4b66006d-d0a6-485c-9f82-343b1d11f776-kube-api-access-8mjg2\") pod \"4b66006d-d0a6-485c-9f82-343b1d11f776\" (UID: \"4b66006d-d0a6-485c-9f82-343b1d11f776\") " Jul 24 13:57:49.128015 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.127954 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b66006d-d0a6-485c-9f82-343b1d11f776-kube-api-access-8mjg2" (OuterVolumeSpecName: "kube-api-access-8mjg2") pod "4b66006d-d0a6-485c-9f82-343b1d11f776" (UID: "4b66006d-d0a6-485c-9f82-343b1d11f776"). InnerVolumeSpecName "kube-api-access-8mjg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:49.226446 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.226363 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mjg2\" (UniqueName: \"kubernetes.io/projected/4b66006d-d0a6-485c-9f82-343b1d11f776-kube-api-access-8mjg2\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:57:49.654429 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.654391 2555 generic.go:356] "Generic (PLEG): container finished" podID="4b66006d-d0a6-485c-9f82-343b1d11f776" containerID="c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117" exitCode=0 Jul 24 13:57:49.654429 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.654430 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mztmg" Jul 24 13:57:49.654949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.654468 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mztmg" event={"ID":"4b66006d-d0a6-485c-9f82-343b1d11f776","Type":"ContainerDied","Data":"c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117"} Jul 24 13:57:49.654949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.654499 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mztmg" event={"ID":"4b66006d-d0a6-485c-9f82-343b1d11f776","Type":"ContainerDied","Data":"9b3860f2171831bd8fef086ba5bcc9ffff7edd41a0147ca4ca371b530f6ccf90"} Jul 24 13:57:49.654949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.654515 2555 scope.go:117] "RemoveContainer" containerID="c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117" Jul 24 13:57:49.663429 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.663410 2555 scope.go:117] "RemoveContainer" containerID="c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117" Jul 24 13:57:49.663681 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:57:49.663664 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117\": container with ID starting with c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117 not found: ID does not exist" containerID="c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117" Jul 24 13:57:49.663725 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.663689 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117"} err="failed to get container status \"c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117\": rpc error: code = NotFound desc = could not find container \"c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117\": container with ID starting with c3226b27ea2fed7a517d4bc7d00cef3545b2fe438ebdf2918de533481298c117 not found: ID does not exist" Jul 24 13:57:49.675478 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.675449 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mztmg"] Jul 24 13:57:49.677914 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.677891 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mztmg"] Jul 24 13:57:49.749464 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.749433 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b66006d-d0a6-485c-9f82-343b1d11f776" path="/var/lib/kubelet/pods/4b66006d-d0a6-485c-9f82-343b1d11f776/volumes" Jul 24 13:57:49.749745 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:49.749734 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a93d182c-6795-4603-9b70-d5ad8b30cd98" path="/var/lib/kubelet/pods/a93d182c-6795-4603-9b70-d5ad8b30cd98/volumes" Jul 24 13:57:53.523018 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.522906 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:53.538555 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.538519 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-856486d58-6d799"] Jul 24 13:57:53.538869 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.538857 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b66006d-d0a6-485c-9f82-343b1d11f776" containerName="authorino" Jul 24 13:57:53.538916 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.538871 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b66006d-d0a6-485c-9f82-343b1d11f776" containerName="authorino" Jul 24 13:57:53.538949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.538941 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b66006d-d0a6-485c-9f82-343b1d11f776" containerName="authorino" Jul 24 13:57:53.543570 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.543544 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.548688 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.548667 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:57:53.549888 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.549869 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jjwlm\"" Jul 24 13:57:53.556818 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.556797 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:57:53.560143 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.560115 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-856486d58-6d799"] Jul 24 13:57:53.640808 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.640773 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:53.665638 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.665599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/916f7386-b159-48ba-84ad-f70dc2d410b7-webhook-cert\") pod \"maas-controller-856486d58-6d799\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.665803 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.665653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9kpt\" (UniqueName: \"kubernetes.io/projected/916f7386-b159-48ba-84ad-f70dc2d410b7-kube-api-access-j9kpt\") pod \"maas-controller-856486d58-6d799\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.724888 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.724860 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:53.766140 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.766108 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9kpt\" (UniqueName: \"kubernetes.io/projected/916f7386-b159-48ba-84ad-f70dc2d410b7-kube-api-access-j9kpt\") pod \"maas-controller-856486d58-6d799\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.766333 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.766221 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/916f7386-b159-48ba-84ad-f70dc2d410b7-webhook-cert\") pod \"maas-controller-856486d58-6d799\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.768810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.768786 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/916f7386-b159-48ba-84ad-f70dc2d410b7-webhook-cert\") pod \"maas-controller-856486d58-6d799\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.779181 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.779120 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9kpt\" (UniqueName: \"kubernetes.io/projected/916f7386-b159-48ba-84ad-f70dc2d410b7-kube-api-access-j9kpt\") pod \"maas-controller-856486d58-6d799\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.825431 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.825401 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:53.854465 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.854433 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:53.997610 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:53.997583 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-856486d58-6d799"] Jul 24 13:57:54.000021 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:57:53.999961 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916f7386_b159_48ba_84ad_f70dc2d410b7.slice/crio-a9e786d163a04597b5201672be08ff494b0ebd30cd6b15e5b56e804c9fda5383 WatchSource:0}: Error finding container a9e786d163a04597b5201672be08ff494b0ebd30cd6b15e5b56e804c9fda5383: Status 404 returned error can't find the container with id a9e786d163a04597b5201672be08ff494b0ebd30cd6b15e5b56e804c9fda5383 Jul 24 13:57:54.680524 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:54.680482 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-856486d58-6d799" event={"ID":"916f7386-b159-48ba-84ad-f70dc2d410b7","Type":"ContainerStarted","Data":"a9e786d163a04597b5201672be08ff494b0ebd30cd6b15e5b56e804c9fda5383"} Jul 24 13:57:56.625485 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:56.625449 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:57:56.692695 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:56.692602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-856486d58-6d799" event={"ID":"916f7386-b159-48ba-84ad-f70dc2d410b7","Type":"ContainerStarted","Data":"e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7"} Jul 24 13:57:56.692871 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:56.692722 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:57:56.756005 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:57:56.755948 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:07.700277 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:07.700249 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:58:07.721511 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:07.721461 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-856486d58-6d799" podStartSLOduration=12.280606174 podStartE2EDuration="14.721443726s" totalImagesPullingTime="2.440837552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:57:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:57:56.750529102 +0000 UTC m=+763.545871833" watchObservedRunningTime="2026-07-24 13:58:07.721443726 +0000 UTC m=+774.516786456" Jul 24 13:58:08.151708 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.151672 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-856486d58-6d799"] Jul 24 13:58:08.151885 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.151861 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-856486d58-6d799" podUID="916f7386-b159-48ba-84ad-f70dc2d410b7" containerName="manager" containerID="cri-o://e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7" gracePeriod=30 Jul 24 13:58:08.408341 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.408277 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:58:08.493697 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.493663 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9kpt\" (UniqueName: \"kubernetes.io/projected/916f7386-b159-48ba-84ad-f70dc2d410b7-kube-api-access-j9kpt\") pod \"916f7386-b159-48ba-84ad-f70dc2d410b7\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " Jul 24 13:58:08.493941 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.493714 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/916f7386-b159-48ba-84ad-f70dc2d410b7-webhook-cert\") pod \"916f7386-b159-48ba-84ad-f70dc2d410b7\" (UID: \"916f7386-b159-48ba-84ad-f70dc2d410b7\") " Jul 24 13:58:08.496005 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.495964 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/916f7386-b159-48ba-84ad-f70dc2d410b7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "916f7386-b159-48ba-84ad-f70dc2d410b7" (UID: "916f7386-b159-48ba-84ad-f70dc2d410b7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:58:08.496094 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.496049 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/916f7386-b159-48ba-84ad-f70dc2d410b7-kube-api-access-j9kpt" (OuterVolumeSpecName: "kube-api-access-j9kpt") pod "916f7386-b159-48ba-84ad-f70dc2d410b7" (UID: "916f7386-b159-48ba-84ad-f70dc2d410b7"). InnerVolumeSpecName "kube-api-access-j9kpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:58:08.594613 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.594567 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/916f7386-b159-48ba-84ad-f70dc2d410b7-webhook-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:58:08.594613 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.594603 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9kpt\" (UniqueName: \"kubernetes.io/projected/916f7386-b159-48ba-84ad-f70dc2d410b7-kube-api-access-j9kpt\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 13:58:08.748961 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.748871 2555 generic.go:356] "Generic (PLEG): container finished" podID="916f7386-b159-48ba-84ad-f70dc2d410b7" containerID="e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7" exitCode=0 Jul 24 13:58:08.748961 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.748913 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-856486d58-6d799" Jul 24 13:58:08.749476 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.748955 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-856486d58-6d799" event={"ID":"916f7386-b159-48ba-84ad-f70dc2d410b7","Type":"ContainerDied","Data":"e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7"} Jul 24 13:58:08.749476 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.749025 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-856486d58-6d799" event={"ID":"916f7386-b159-48ba-84ad-f70dc2d410b7","Type":"ContainerDied","Data":"a9e786d163a04597b5201672be08ff494b0ebd30cd6b15e5b56e804c9fda5383"} Jul 24 13:58:08.749476 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.749049 2555 scope.go:117] "RemoveContainer" containerID="e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7" Jul 24 13:58:08.758940 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.758915 2555 scope.go:117] "RemoveContainer" containerID="e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7" Jul 24 13:58:08.759274 ip-10-0-143-204 kubenswrapper[2555]: E0724 13:58:08.759255 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7\": container with ID starting with e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7 not found: ID does not exist" containerID="e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7" Jul 24 13:58:08.759332 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.759284 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7"} err="failed to get container status \"e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7\": rpc error: code = NotFound desc = could not find container \"e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7\": container with ID starting with e820586e22cd21fcf0881ba9ba4f82a3f01b8f0816c6f79d3c09400f5fdc05e7 not found: ID does not exist" Jul 24 13:58:08.772878 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.772848 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-856486d58-6d799"] Jul 24 13:58:08.777647 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:08.777623 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-856486d58-6d799"] Jul 24 13:58:09.750164 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:09.750132 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="916f7386-b159-48ba-84ad-f70dc2d410b7" path="/var/lib/kubelet/pods/916f7386-b159-48ba-84ad-f70dc2d410b7/volumes" Jul 24 13:58:28.490194 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:28.490163 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:28.617958 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:28.617923 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:29.121743 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.121707 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:29.216447 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.216418 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:29.677425 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.677395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:29.757094 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.757061 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:29.757644 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.757625 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z"] Jul 24 13:58:29.758018 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.758003 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="916f7386-b159-48ba-84ad-f70dc2d410b7" containerName="manager" Jul 24 13:58:29.758069 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.758020 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="916f7386-b159-48ba-84ad-f70dc2d410b7" containerName="manager" Jul 24 13:58:29.758109 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.758082 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="916f7386-b159-48ba-84ad-f70dc2d410b7" containerName="manager" Jul 24 13:58:29.763340 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.763319 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:29.765837 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.765816 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 13:58:29.772272 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.772235 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z"] Jul 24 13:58:29.794996 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.794959 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-nftmt"] Jul 24 13:58:29.797544 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.797525 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.800738 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.800716 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bsrdb\"" Jul 24 13:58:29.809746 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.809716 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-nftmt"] Jul 24 13:58:29.879706 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.879674 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tglfl\" (UniqueName: \"kubernetes.io/projected/ac51fa65-d5d0-40ae-8e74-53802b5bc606-kube-api-access-tglfl\") pod \"payload-pre-processing-687dbb6cf6-rsm9z\" (UID: \"ac51fa65-d5d0-40ae-8e74-53802b5bc606\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:29.879882 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.879746 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac51fa65-d5d0-40ae-8e74-53802b5bc606-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-rsm9z\" (UID: \"ac51fa65-d5d0-40ae-8e74-53802b5bc606\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:29.879882 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.879790 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n79hc\" (UniqueName: \"kubernetes.io/projected/02b5d03f-8032-42aa-ae94-c3521ea07a7d-kube-api-access-n79hc\") pod \"payload-processing-5cc6cd444b-nftmt\" (UID: \"02b5d03f-8032-42aa-ae94-c3521ea07a7d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.879882 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.879824 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b5d03f-8032-42aa-ae94-c3521ea07a7d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-nftmt\" (UID: \"02b5d03f-8032-42aa-ae94-c3521ea07a7d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.981229 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.981150 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac51fa65-d5d0-40ae-8e74-53802b5bc606-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-rsm9z\" (UID: \"ac51fa65-d5d0-40ae-8e74-53802b5bc606\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:29.981229 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.981204 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n79hc\" (UniqueName: \"kubernetes.io/projected/02b5d03f-8032-42aa-ae94-c3521ea07a7d-kube-api-access-n79hc\") pod \"payload-processing-5cc6cd444b-nftmt\" (UID: \"02b5d03f-8032-42aa-ae94-c3521ea07a7d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.981455 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.981234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b5d03f-8032-42aa-ae94-c3521ea07a7d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-nftmt\" (UID: \"02b5d03f-8032-42aa-ae94-c3521ea07a7d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.981455 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.981297 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tglfl\" (UniqueName: \"kubernetes.io/projected/ac51fa65-d5d0-40ae-8e74-53802b5bc606-kube-api-access-tglfl\") pod \"payload-pre-processing-687dbb6cf6-rsm9z\" (UID: \"ac51fa65-d5d0-40ae-8e74-53802b5bc606\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:29.981861 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.981833 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac51fa65-d5d0-40ae-8e74-53802b5bc606-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-rsm9z\" (UID: \"ac51fa65-d5d0-40ae-8e74-53802b5bc606\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:29.981974 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.981964 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02b5d03f-8032-42aa-ae94-c3521ea07a7d-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-nftmt\" (UID: \"02b5d03f-8032-42aa-ae94-c3521ea07a7d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.989773 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.989739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n79hc\" (UniqueName: \"kubernetes.io/projected/02b5d03f-8032-42aa-ae94-c3521ea07a7d-kube-api-access-n79hc\") pod \"payload-processing-5cc6cd444b-nftmt\" (UID: \"02b5d03f-8032-42aa-ae94-c3521ea07a7d\") " pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:29.989879 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:29.989749 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tglfl\" (UniqueName: \"kubernetes.io/projected/ac51fa65-d5d0-40ae-8e74-53802b5bc606-kube-api-access-tglfl\") pod \"payload-pre-processing-687dbb6cf6-rsm9z\" (UID: \"ac51fa65-d5d0-40ae-8e74-53802b5bc606\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:30.076091 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:30.076055 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:30.110474 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:30.110441 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:30.238852 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:30.238791 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z"] Jul 24 13:58:30.241315 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:58:30.241286 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac51fa65_d5d0_40ae_8e74_53802b5bc606.slice/crio-84a52c1fc1f11d411f7df6bd9791c940bb2ecd760d42dc15a5d9efdcefab0fd5 WatchSource:0}: Error finding container 84a52c1fc1f11d411f7df6bd9791c940bb2ecd760d42dc15a5d9efdcefab0fd5: Status 404 returned error can't find the container with id 84a52c1fc1f11d411f7df6bd9791c940bb2ecd760d42dc15a5d9efdcefab0fd5 Jul 24 13:58:30.263554 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:30.263530 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-nftmt"] Jul 24 13:58:30.265617 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:58:30.265590 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02b5d03f_8032_42aa_ae94_c3521ea07a7d.slice/crio-106b66d73c189c5839422cad82039cd96dc339c9418bbc67829d7c939d54ad02 WatchSource:0}: Error finding container 106b66d73c189c5839422cad82039cd96dc339c9418bbc67829d7c939d54ad02: Status 404 returned error can't find the container with id 106b66d73c189c5839422cad82039cd96dc339c9418bbc67829d7c939d54ad02 Jul 24 13:58:30.853533 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:30.853455 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" event={"ID":"02b5d03f-8032-42aa-ae94-c3521ea07a7d","Type":"ContainerStarted","Data":"106b66d73c189c5839422cad82039cd96dc339c9418bbc67829d7c939d54ad02"} Jul 24 13:58:30.855400 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:30.855282 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" event={"ID":"ac51fa65-d5d0-40ae-8e74-53802b5bc606","Type":"ContainerStarted","Data":"84a52c1fc1f11d411f7df6bd9791c940bb2ecd760d42dc15a5d9efdcefab0fd5"} Jul 24 13:58:32.867762 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:32.867720 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" event={"ID":"ac51fa65-d5d0-40ae-8e74-53802b5bc606","Type":"ContainerStarted","Data":"cb978f437d96b7e02cab5588f173d0beb402890a1305835447cbd9f4dbf04a46"} Jul 24 13:58:32.868271 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:32.867787 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:32.869290 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:32.869264 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" event={"ID":"02b5d03f-8032-42aa-ae94-c3521ea07a7d","Type":"ContainerStarted","Data":"6206af15aeeb6501ad0f2a05a00beede6a627fd26e77190ad423f4cedf4b02a7"} Jul 24 13:58:32.869405 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:32.869390 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:32.888454 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:32.888391 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" podStartSLOduration=1.7272484769999998 podStartE2EDuration="3.888367197s" totalImagesPullingTime="2.16111872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:58:32.885810969 +0000 UTC m=+799.681153702" watchObservedRunningTime="2026-07-24 13:58:32.888367197 +0000 UTC m=+799.683709928" Jul 24 13:58:32.921772 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:32.921706 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" podStartSLOduration=1.812573735 podStartE2EDuration="3.921682591s" totalImagesPullingTime="2.109108856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:58:32.904511488 +0000 UTC m=+799.699854209" watchObservedRunningTime="2026-07-24 13:58:32.921682591 +0000 UTC m=+799.717025327" Jul 24 13:58:37.895153 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:37.895121 2555 generic.go:356] "Generic (PLEG): container finished" podID="dd838d0a-45bf-4e91-8473-41bb74497fc1" containerID="97109490e34f4706a4d5ecde8cefc3986ea33365f5fe6299939f193bf75a59f8" exitCode=0 Jul 24 13:58:37.895634 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:37.895193 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" event={"ID":"dd838d0a-45bf-4e91-8473-41bb74497fc1","Type":"ContainerDied","Data":"97109490e34f4706a4d5ecde8cefc3986ea33365f5fe6299939f193bf75a59f8"} Jul 24 13:58:37.895634 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:37.895572 2555 scope.go:117] "RemoveContainer" containerID="97109490e34f4706a4d5ecde8cefc3986ea33365f5fe6299939f193bf75a59f8" Jul 24 13:58:37.896608 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:37.896575 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:58:37.896726 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:37.896667 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:58:37.896726 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:37.896713 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 13:58:38.905393 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:38.905347 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" event={"ID":"dd838d0a-45bf-4e91-8473-41bb74497fc1","Type":"ContainerStarted","Data":"95427564cf1535f8eb83e76f9ae6cfc90fff6d903011b437fde7746744763629"} Jul 24 13:58:39.046172 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:39.046130 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:58:39.046356 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:39.046331 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qkcrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 13:58:39.046402 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:39.046383 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" podUID="dd838d0a-45bf-4e91-8473-41bb74497fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 13:58:39.933903 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:39.933865 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:40.038551 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:40.038509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:40.046998 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:40.046960 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qkcrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 13:58:40.047084 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:40.047035 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" podUID="dd838d0a-45bf-4e91-8473-41bb74497fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 13:58:41.046678 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:41.046646 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qkcrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 13:58:41.047120 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:41.046701 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" podUID="dd838d0a-45bf-4e91-8473-41bb74497fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 13:58:42.046999 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:42.046947 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-qkcrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 24 13:58:42.047407 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:42.047023 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" podUID="dd838d0a-45bf-4e91-8473-41bb74497fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 24 13:58:43.087937 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:43.087898 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:58:43.088350 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:43.088213 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:58:43.088996 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:43.088962 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-qkcrn" Jul 24 13:58:43.876763 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:43.876727 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-rsm9z" Jul 24 13:58:43.876959 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:43.876905 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-nftmt" Jul 24 13:58:53.647069 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.647035 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz"] Jul 24 13:58:53.649810 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.649787 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.653883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.653850 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 24 13:58:53.653883 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.653876 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 13:58:53.654115 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.653917 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 13:58:53.654115 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.653877 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xl92g\"" Jul 24 13:58:53.660315 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.660288 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz"] Jul 24 13:58:53.672528 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.672477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:58:53.710559 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.710525 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.710775 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.710585 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75kkn\" (UniqueName: \"kubernetes.io/projected/9d39bf1f-d096-45b8-b980-02b367cd0857-kube-api-access-75kkn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.710775 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.710658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.710775 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.710716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d39bf1f-d096-45b8-b980-02b367cd0857-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.710956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.710824 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.710956 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.710878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812311 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812281 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812325 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75kkn\" (UniqueName: \"kubernetes.io/projected/9d39bf1f-d096-45b8-b980-02b367cd0857-kube-api-access-75kkn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812498 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812413 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d39bf1f-d096-45b8-b980-02b367cd0857-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812790 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812924 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812850 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.812924 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.812877 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.814949 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.814926 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d39bf1f-d096-45b8-b980-02b367cd0857-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.815327 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.815310 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d39bf1f-d096-45b8-b980-02b367cd0857-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.823047 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.823022 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75kkn\" (UniqueName: \"kubernetes.io/projected/9d39bf1f-d096-45b8-b980-02b367cd0857-kube-api-access-75kkn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz\" (UID: \"9d39bf1f-d096-45b8-b980-02b367cd0857\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:53.961234 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:53.961138 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:58:54.113251 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:54.113153 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz"] Jul 24 13:58:54.116873 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:58:54.116847 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d39bf1f_d096_45b8_b980_02b367cd0857.slice/crio-a5c099901c768e0de4ec4955dc31208646f57bf45c38ae4ecf5410eb72600a93 WatchSource:0}: Error finding container a5c099901c768e0de4ec4955dc31208646f57bf45c38ae4ecf5410eb72600a93: Status 404 returned error can't find the container with id a5c099901c768e0de4ec4955dc31208646f57bf45c38ae4ecf5410eb72600a93 Jul 24 13:58:54.988654 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:54.988622 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" event={"ID":"9d39bf1f-d096-45b8-b980-02b367cd0857","Type":"ContainerStarted","Data":"a5c099901c768e0de4ec4955dc31208646f57bf45c38ae4ecf5410eb72600a93"} Jul 24 13:58:55.994959 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:55.994921 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" event={"ID":"9d39bf1f-d096-45b8-b980-02b367cd0857","Type":"ContainerStarted","Data":"faff04c9f13c93cb06100760fb2736ce1516385c5716723f508dfc2e2b898ab8"} Jul 24 13:58:56.013838 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:58:56.013786 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" podStartSLOduration=1.8374951670000002 podStartE2EDuration="3.013767953s" totalImagesPullingTime="1.176272786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:58:56.011789954 +0000 UTC m=+822.807132722" watchObservedRunningTime="2026-07-24 13:58:56.013767953 +0000 UTC m=+822.809110725" Jul 24 13:59:03.961652 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:03.961606 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:59:03.962252 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:03.961676 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:59:03.974462 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:03.974436 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:59:04.048657 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:04.048627 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz" Jul 24 13:59:05.361666 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:05.361629 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:24.562541 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.562497 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d"] Jul 24 13:59:24.565396 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.565377 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.568303 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.568282 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 24 13:59:24.576137 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.576112 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d"] Jul 24 13:59:24.607180 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.607151 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:24.701224 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.701188 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.701224 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.701222 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.701449 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.701281 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a88726-271a-4df2-9b87-4431f7462a0d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.701449 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.701347 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.701449 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.701365 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp7pt\" (UniqueName: \"kubernetes.io/projected/d9a88726-271a-4df2-9b87-4431f7462a0d-kube-api-access-jp7pt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.701449 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.701387 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.802898 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.802860 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.802898 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.802898 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp7pt\" (UniqueName: \"kubernetes.io/projected/d9a88726-271a-4df2-9b87-4431f7462a0d-kube-api-access-jp7pt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.802922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.802958 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.803010 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803177 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.803069 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a88726-271a-4df2-9b87-4431f7462a0d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803432 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.803409 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803507 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.803448 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.803507 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.803481 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.805270 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.805250 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9a88726-271a-4df2-9b87-4431f7462a0d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.805605 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.805589 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9a88726-271a-4df2-9b87-4431f7462a0d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.816888 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.816827 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp7pt\" (UniqueName: \"kubernetes.io/projected/d9a88726-271a-4df2-9b87-4431f7462a0d-kube-api-access-jp7pt\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zh54d\" (UID: \"d9a88726-271a-4df2-9b87-4431f7462a0d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:24.876940 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:24.876904 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:25.015744 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:25.015718 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d"] Jul 24 13:59:25.017645 ip-10-0-143-204 kubenswrapper[2555]: W0724 13:59:25.017618 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a88726_271a_4df2_9b87_4431f7462a0d.slice/crio-f859d1319ecf6567f06259fd7d435e29bacb90cebfb2ef18a84d850baf1d9757 WatchSource:0}: Error finding container f859d1319ecf6567f06259fd7d435e29bacb90cebfb2ef18a84d850baf1d9757: Status 404 returned error can't find the container with id f859d1319ecf6567f06259fd7d435e29bacb90cebfb2ef18a84d850baf1d9757 Jul 24 13:59:25.136130 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:25.136046 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" event={"ID":"d9a88726-271a-4df2-9b87-4431f7462a0d","Type":"ContainerStarted","Data":"f859d1319ecf6567f06259fd7d435e29bacb90cebfb2ef18a84d850baf1d9757"} Jul 24 13:59:26.143506 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:26.143469 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" event={"ID":"d9a88726-271a-4df2-9b87-4431f7462a0d","Type":"ContainerStarted","Data":"ec45f9ce79007222c2d21991e09a449eddd5295a5818b6051299b32b5ce5b280"} Jul 24 13:59:26.160455 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:26.160398 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" podStartSLOduration=1.950758683 podStartE2EDuration="2.160380032s" totalImagesPullingTime="209.621349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:59:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:59:26.159854127 +0000 UTC m=+852.955196858" watchObservedRunningTime="2026-07-24 13:59:26.160380032 +0000 UTC m=+852.955722810" Jul 24 13:59:34.877118 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:34.877060 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:34.877657 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:34.877182 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:34.889736 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:34.889698 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:35.203232 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:35.203142 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zh54d" Jul 24 13:59:35.647767 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:35.647717 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:38.235534 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:38.235489 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 13:59:55.995347 ip-10-0-143-204 kubenswrapper[2555]: I0724 13:59:55.995315 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:23.115698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:23.115666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:23.174898 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:23.174862 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:53.265468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.265430 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6fb7b944c7-vjs5r"] Jul 24 14:00:53.268746 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.268729 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.276435 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.276411 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fb7b944c7-vjs5r"] Jul 24 14:00:53.406380 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.406335 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl6jz\" (UniqueName: \"kubernetes.io/projected/ed8af88c-6c72-4b23-8594-19e694e41072-kube-api-access-cl6jz\") pod \"authorino-6fb7b944c7-vjs5r\" (UID: \"ed8af88c-6c72-4b23-8594-19e694e41072\") " pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.406563 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.406402 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed8af88c-6c72-4b23-8594-19e694e41072-tls-cert\") pod \"authorino-6fb7b944c7-vjs5r\" (UID: \"ed8af88c-6c72-4b23-8594-19e694e41072\") " pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.507357 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.507318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl6jz\" (UniqueName: \"kubernetes.io/projected/ed8af88c-6c72-4b23-8594-19e694e41072-kube-api-access-cl6jz\") pod \"authorino-6fb7b944c7-vjs5r\" (UID: \"ed8af88c-6c72-4b23-8594-19e694e41072\") " pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.507530 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.507375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed8af88c-6c72-4b23-8594-19e694e41072-tls-cert\") pod \"authorino-6fb7b944c7-vjs5r\" (UID: \"ed8af88c-6c72-4b23-8594-19e694e41072\") " pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.510192 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.510158 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ed8af88c-6c72-4b23-8594-19e694e41072-tls-cert\") pod \"authorino-6fb7b944c7-vjs5r\" (UID: \"ed8af88c-6c72-4b23-8594-19e694e41072\") " pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.515825 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.515756 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl6jz\" (UniqueName: \"kubernetes.io/projected/ed8af88c-6c72-4b23-8594-19e694e41072-kube-api-access-cl6jz\") pod \"authorino-6fb7b944c7-vjs5r\" (UID: \"ed8af88c-6c72-4b23-8594-19e694e41072\") " pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.578418 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.578384 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" Jul 24 14:00:53.709150 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.709121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:53.721698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.721677 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6fb7b944c7-vjs5r"] Jul 24 14:00:53.724179 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:00:53.724152 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded8af88c_6c72_4b23_8594_19e694e41072.slice/crio-f02624387a24b46f7ae5272bd9c925d2768b31b2b77288693e01d3d599d75956 WatchSource:0}: Error finding container f02624387a24b46f7ae5272bd9c925d2768b31b2b77288693e01d3d599d75956: Status 404 returned error can't find the container with id f02624387a24b46f7ae5272bd9c925d2768b31b2b77288693e01d3d599d75956 Jul 24 14:00:53.725792 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.725777 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 14:00:53.808602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:53.808568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:54.581503 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.581463 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" event={"ID":"ed8af88c-6c72-4b23-8594-19e694e41072","Type":"ContainerStarted","Data":"b2afae1fe893f6f880ec4cf43176d48c220355b236755e2eb49e9e3570dadbf3"} Jul 24 14:00:54.582020 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.581512 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" event={"ID":"ed8af88c-6c72-4b23-8594-19e694e41072","Type":"ContainerStarted","Data":"f02624387a24b46f7ae5272bd9c925d2768b31b2b77288693e01d3d599d75956"} Jul 24 14:00:54.600671 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.600604 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6fb7b944c7-vjs5r" podStartSLOduration=1.15670043 podStartE2EDuration="1.600579136s" totalImagesPullingTime="443.878706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:00:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:00:54.599706699 +0000 UTC m=+941.395049421" watchObservedRunningTime="2026-07-24 14:00:54.600579136 +0000 UTC m=+941.395921868" Jul 24 14:00:54.630654 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.630616 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c4f568fc6-wd884"] Jul 24 14:00:54.630856 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.630831 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c4f568fc6-wd884" podUID="55864e0e-4647-465e-8f7f-dd1ee178e170" containerName="authorino" containerID="cri-o://afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e" gracePeriod=30 Jul 24 14:00:54.815787 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.815752 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:54.900755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.900729 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 14:00:54.913417 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:54.913379 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:00:55.022935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.022901 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkjlw\" (UniqueName: \"kubernetes.io/projected/55864e0e-4647-465e-8f7f-dd1ee178e170-kube-api-access-gkjlw\") pod \"55864e0e-4647-465e-8f7f-dd1ee178e170\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " Jul 24 14:00:55.022935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.022940 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55864e0e-4647-465e-8f7f-dd1ee178e170-tls-cert\") pod \"55864e0e-4647-465e-8f7f-dd1ee178e170\" (UID: \"55864e0e-4647-465e-8f7f-dd1ee178e170\") " Jul 24 14:00:55.025353 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.025316 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55864e0e-4647-465e-8f7f-dd1ee178e170-kube-api-access-gkjlw" (OuterVolumeSpecName: "kube-api-access-gkjlw") pod "55864e0e-4647-465e-8f7f-dd1ee178e170" (UID: "55864e0e-4647-465e-8f7f-dd1ee178e170"). InnerVolumeSpecName "kube-api-access-gkjlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:00:55.035312 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.035268 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55864e0e-4647-465e-8f7f-dd1ee178e170-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "55864e0e-4647-465e-8f7f-dd1ee178e170" (UID: "55864e0e-4647-465e-8f7f-dd1ee178e170"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 14:00:55.124031 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.123973 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkjlw\" (UniqueName: \"kubernetes.io/projected/55864e0e-4647-465e-8f7f-dd1ee178e170-kube-api-access-gkjlw\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:00:55.124031 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.124031 2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/55864e0e-4647-465e-8f7f-dd1ee178e170-tls-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:00:55.588784 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.588746 2555 generic.go:356] "Generic (PLEG): container finished" podID="55864e0e-4647-465e-8f7f-dd1ee178e170" containerID="afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e" exitCode=0 Jul 24 14:00:55.589287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.588801 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c4f568fc6-wd884" Jul 24 14:00:55.589287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.588831 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4f568fc6-wd884" event={"ID":"55864e0e-4647-465e-8f7f-dd1ee178e170","Type":"ContainerDied","Data":"afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e"} Jul 24 14:00:55.589287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.588886 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c4f568fc6-wd884" event={"ID":"55864e0e-4647-465e-8f7f-dd1ee178e170","Type":"ContainerDied","Data":"ca08e90b318726736bcda2d2e5f8acaf50cae0f4730293e67ed2fe42e38c6ce3"} Jul 24 14:00:55.589287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.588910 2555 scope.go:117] "RemoveContainer" containerID="afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e" Jul 24 14:00:55.600589 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.600565 2555 scope.go:117] "RemoveContainer" containerID="afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e" Jul 24 14:00:55.600887 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:00:55.600866 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e\": container with ID starting with afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e not found: ID does not exist" containerID="afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e" Jul 24 14:00:55.600954 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.600896 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e"} err="failed to get container status \"afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e\": rpc error: code = NotFound desc = could not find container \"afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e\": container with ID starting with afe0f3c2e1f26687bf214272229072c5fcdee92faf2edfedc3e1d1914345073e not found: ID does not exist" Jul 24 14:00:55.611274 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.611241 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c4f568fc6-wd884"] Jul 24 14:00:55.615263 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.615232 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c4f568fc6-wd884"] Jul 24 14:00:55.750046 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:00:55.750016 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55864e0e-4647-465e-8f7f-dd1ee178e170" path="/var/lib/kubelet/pods/55864e0e-4647-465e-8f7f-dd1ee178e170/volumes" Jul 24 14:01:12.818052 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:01:12.818008 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:28.599694 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:01:28.599658 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:01:47.710852 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:01:47.710751 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:02:59.150250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:02:59.150208 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:00.408790 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:00.408755 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:34.052119 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:34.052082 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:34.152726 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:34.152683 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:34.322096 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:34.321995 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:34.696699 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:34.696597 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:34.791190 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:34.791160 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:40.069800 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:40.069765 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:40.137352 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:40.137317 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:40.246959 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:40.246923 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:40.345677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:40.345566 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:40.477737 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:40.477689 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:46.331100 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:46.331063 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:46.403592 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:46.403540 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:55.452192 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:55.452156 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:58.751166 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:58.751127 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:03:58.851391 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:03:58.851351 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:04:25.942922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:04:25.942888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:30.761070 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:30.761038 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:30.847075 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:30.847033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:07:47.867550 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:47.867461 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g"] Jul 24 14:07:47.867969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:47.867717 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" podUID="13b1e7ea-a99e-4513-b6c2-43b59a261982" containerName="manager" containerID="cri-o://d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7" gracePeriod=10 Jul 24 14:07:51.523650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.523625 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 14:07:51.616444 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.616401 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b1e7ea-a99e-4513-b6c2-43b59a261982-extensions-socket-volume\") pod \"13b1e7ea-a99e-4513-b6c2-43b59a261982\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " Jul 24 14:07:51.616634 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.616522 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmnvg\" (UniqueName: \"kubernetes.io/projected/13b1e7ea-a99e-4513-b6c2-43b59a261982-kube-api-access-nmnvg\") pod \"13b1e7ea-a99e-4513-b6c2-43b59a261982\" (UID: \"13b1e7ea-a99e-4513-b6c2-43b59a261982\") " Jul 24 14:07:51.616887 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.616856 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13b1e7ea-a99e-4513-b6c2-43b59a261982-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "13b1e7ea-a99e-4513-b6c2-43b59a261982" (UID: "13b1e7ea-a99e-4513-b6c2-43b59a261982"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:07:51.618795 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.618763 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b1e7ea-a99e-4513-b6c2-43b59a261982-kube-api-access-nmnvg" (OuterVolumeSpecName: "kube-api-access-nmnvg") pod "13b1e7ea-a99e-4513-b6c2-43b59a261982" (UID: "13b1e7ea-a99e-4513-b6c2-43b59a261982"). InnerVolumeSpecName "kube-api-access-nmnvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:07:51.634388 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.634354 2555 generic.go:356] "Generic (PLEG): container finished" podID="13b1e7ea-a99e-4513-b6c2-43b59a261982" containerID="d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7" exitCode=0 Jul 24 14:07:51.634555 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.634404 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" Jul 24 14:07:51.634555 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.634443 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" event={"ID":"13b1e7ea-a99e-4513-b6c2-43b59a261982","Type":"ContainerDied","Data":"d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7"} Jul 24 14:07:51.634555 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.634514 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g" event={"ID":"13b1e7ea-a99e-4513-b6c2-43b59a261982","Type":"ContainerDied","Data":"f33c8a6dc9a356acf5cf1c1cfb5db2e015ab02240e1cdcdd97c3b2baa310b07c"} Jul 24 14:07:51.634555 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.634538 2555 scope.go:117] "RemoveContainer" containerID="d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7" Jul 24 14:07:51.648459 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.648439 2555 scope.go:117] "RemoveContainer" containerID="d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7" Jul 24 14:07:51.648748 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:07:51.648728 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7\": container with ID starting with d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7 not found: ID does not exist" containerID="d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7" Jul 24 14:07:51.648835 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.648759 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7"} err="failed to get container status \"d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7\": rpc error: code = NotFound desc = could not find container \"d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7\": container with ID starting with d2bb0efd04c997c91be0582fa9cbe1f9b726b92580a89fdd1cab9d4ce105c7d7 not found: ID does not exist" Jul 24 14:07:51.658176 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.658149 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g"] Jul 24 14:07:51.664325 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.664300 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jx44g"] Jul 24 14:07:51.717131 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.717097 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmnvg\" (UniqueName: \"kubernetes.io/projected/13b1e7ea-a99e-4513-b6c2-43b59a261982-kube-api-access-nmnvg\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:07:51.717131 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.717123 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13b1e7ea-a99e-4513-b6c2-43b59a261982-extensions-socket-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:07:51.750648 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:07:51.750607 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b1e7ea-a99e-4513-b6c2-43b59a261982" path="/var/lib/kubelet/pods/13b1e7ea-a99e-4513-b6c2-43b59a261982/volumes" Jul 24 14:08:42.800590 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:42.800547 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:44.991115 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:44.991074 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:45.046309 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:45.046271 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:08:49.120355 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120314 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n"] Jul 24 14:08:49.120735 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120711 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55864e0e-4647-465e-8f7f-dd1ee178e170" containerName="authorino" Jul 24 14:08:49.120735 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120726 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="55864e0e-4647-465e-8f7f-dd1ee178e170" containerName="authorino" Jul 24 14:08:49.120735 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120734 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b1e7ea-a99e-4513-b6c2-43b59a261982" containerName="manager" Jul 24 14:08:49.120840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120740 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b1e7ea-a99e-4513-b6c2-43b59a261982" containerName="manager" Jul 24 14:08:49.120840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120798 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="55864e0e-4647-465e-8f7f-dd1ee178e170" containerName="authorino" Jul 24 14:08:49.120840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.120812 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b1e7ea-a99e-4513-b6c2-43b59a261982" containerName="manager" Jul 24 14:08:49.123828 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.123810 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.126844 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.126823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fxrj6\"" Jul 24 14:08:49.135015 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.134966 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n"] Jul 24 14:08:49.241307 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.241272 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b5jh\" (UniqueName: \"kubernetes.io/projected/37bcc377-9cb5-4e7a-858b-febfa7ff7681-kube-api-access-8b5jh\") pod \"kuadrant-operator-controller-manager-86878c769f-8r25n\" (UID: \"37bcc377-9cb5-4e7a-858b-febfa7ff7681\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.241505 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.241325 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37bcc377-9cb5-4e7a-858b-febfa7ff7681-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8r25n\" (UID: \"37bcc377-9cb5-4e7a-858b-febfa7ff7681\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.342562 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.342507 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b5jh\" (UniqueName: \"kubernetes.io/projected/37bcc377-9cb5-4e7a-858b-febfa7ff7681-kube-api-access-8b5jh\") pod \"kuadrant-operator-controller-manager-86878c769f-8r25n\" (UID: \"37bcc377-9cb5-4e7a-858b-febfa7ff7681\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.342734 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.342581 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37bcc377-9cb5-4e7a-858b-febfa7ff7681-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8r25n\" (UID: \"37bcc377-9cb5-4e7a-858b-febfa7ff7681\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.342966 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.342945 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/37bcc377-9cb5-4e7a-858b-febfa7ff7681-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8r25n\" (UID: \"37bcc377-9cb5-4e7a-858b-febfa7ff7681\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.351600 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.351574 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b5jh\" (UniqueName: \"kubernetes.io/projected/37bcc377-9cb5-4e7a-858b-febfa7ff7681-kube-api-access-8b5jh\") pod \"kuadrant-operator-controller-manager-86878c769f-8r25n\" (UID: \"37bcc377-9cb5-4e7a-858b-febfa7ff7681\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.434636 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.434545 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.594393 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.594365 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n"] Jul 24 14:08:49.596250 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:08:49.596222 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bcc377_9cb5_4e7a_858b_febfa7ff7681.slice/crio-7d01e4f1ec0762272ae0ef2a8264daed88def59a582a83b928d9255f52b3273c WatchSource:0}: Error finding container 7d01e4f1ec0762272ae0ef2a8264daed88def59a582a83b928d9255f52b3273c: Status 404 returned error can't find the container with id 7d01e4f1ec0762272ae0ef2a8264daed88def59a582a83b928d9255f52b3273c Jul 24 14:08:49.909449 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.909409 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" event={"ID":"37bcc377-9cb5-4e7a-858b-febfa7ff7681","Type":"ContainerStarted","Data":"009e029bff16d3188553ea29541a2207cb210f7aa18cb53b98159ecb59b87402"} Jul 24 14:08:49.909623 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.909467 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" event={"ID":"37bcc377-9cb5-4e7a-858b-febfa7ff7681","Type":"ContainerStarted","Data":"7d01e4f1ec0762272ae0ef2a8264daed88def59a582a83b928d9255f52b3273c"} Jul 24 14:08:49.909623 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.909486 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:08:49.935810 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:08:49.935742 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" podStartSLOduration=0.935723524 podStartE2EDuration="935.723524ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:08:49.934714198 +0000 UTC m=+1416.730056932" watchObservedRunningTime="2026-07-24 14:08:49.935723524 +0000 UTC m=+1416.731066253" Jul 24 14:09:00.916204 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:09:00.916172 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8r25n" Jul 24 14:11:03.864410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:11:03.864327 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:11:31.414026 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:11:31.413962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:11:35.688388 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:11:35.688321 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:24.478133 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:12:24.478041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:25.621412 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:12:25.621363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:45.575611 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:12:45.575566 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:12:45.701744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:12:45.701705 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:15:32.328597 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:15:32.328556 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:15:39.528189 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:15:39.528153 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:03.211416 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:03.211349 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:06.569320 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:06.569281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:08.275328 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:08.275294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:13.076791 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.076749 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:13.249514 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.249479 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:13.349428 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.349340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:13.616906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.616829 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv"] Jul 24 14:16:13.620388 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.620371 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.623248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.623226 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-600fea54-openshift-default-dockercfg-lbgwk\"" Jul 24 14:16:13.632619 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.632592 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv"] Jul 24 14:16:13.647866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.647834 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:13.766631 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766587 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-certs\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.766806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766657 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj2p8\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-kube-api-access-sj2p8\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.766806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-data\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.766806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766737 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-token\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.766806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-credential-socket\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.766806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766802 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-envoy\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.767064 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766835 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-podinfo\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.767064 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766866 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8d6ff-e518-4902-a677-a47ac24dd08e-istiod-ca-cert\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.767064 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.766959 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-socket\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868327 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868237 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-data\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868327 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868279 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-token\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868327 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868305 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-credential-socket\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868351 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-envoy\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868389 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-podinfo\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8d6ff-e518-4902-a677-a47ac24dd08e-istiod-ca-cert\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868459 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-socket\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868504 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-certs\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868574 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj2p8\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-kube-api-access-sj2p8\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868918 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868690 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-data\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.868973 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.868958 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-credential-socket\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.869276 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.869228 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8d6ff-e518-4902-a677-a47ac24dd08e-istiod-ca-cert\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.869276 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.869252 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-certs\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.869402 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.869279 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-socket\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.871072 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.871040 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-podinfo\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.871268 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.871253 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-envoy\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.878162 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.878129 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-token\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.878422 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.878401 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj2p8\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-kube-api-access-sj2p8\") pod \"e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:13.889912 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.889879 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:13.930727 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:13.930690 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:14.079313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:14.079289 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv"] Jul 24 14:16:14.081444 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:16:14.081417 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12f8d6ff_e518_4902_a677_a47ac24dd08e.slice/crio-ee908e1137f74a7b6871437ac747ef19eeafbcd1fcfe12a2b498be0a139c8f3e WatchSource:0}: Error finding container ee908e1137f74a7b6871437ac747ef19eeafbcd1fcfe12a2b498be0a139c8f3e: Status 404 returned error can't find the container with id ee908e1137f74a7b6871437ac747ef19eeafbcd1fcfe12a2b498be0a139c8f3e Jul 24 14:16:14.083688 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:14.083633 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:14.083826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:14.083716 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:14.083826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:14.083760 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:14.088025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:14.087952 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" event={"ID":"12f8d6ff-e518-4902-a677-a47ac24dd08e","Type":"ContainerStarted","Data":"ee908e1137f74a7b6871437ac747ef19eeafbcd1fcfe12a2b498be0a139c8f3e"} Jul 24 14:16:15.094639 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:15.094604 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" event={"ID":"12f8d6ff-e518-4902-a677-a47ac24dd08e","Type":"ContainerStarted","Data":"d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695"} Jul 24 14:16:15.119438 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:15.119375 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podStartSLOduration=2.119359412 podStartE2EDuration="2.119359412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:16:15.116264012 +0000 UTC m=+1861.911606744" watchObservedRunningTime="2026-07-24 14:16:15.119359412 +0000 UTC m=+1861.914702143" Jul 24 14:16:15.931383 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:15.931341 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:15.936409 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:15.936378 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:16.099190 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:16.099160 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:16.100184 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:16.100162 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:16.556340 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:16.556295 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:16.744326 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:16.744292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:16.841023 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:16.840921 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:17.111646 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.111563 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:17.479184 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.479089 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:17.578682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.578649 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:17.597089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.597041 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89"] Jul 24 14:16:17.600194 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.600175 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.603047 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.603019 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-600fea54-dockercfg-stmgl\"" Jul 24 14:16:17.603206 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.603019 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-600fea54\"" Jul 24 14:16:17.608750 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.608721 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89"] Jul 24 14:16:17.625170 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.625133 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst"] Jul 24 14:16:17.630347 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.630316 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.641159 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.641130 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst"] Jul 24 14:16:17.704622 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.704580 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxzz\" (UniqueName: \"kubernetes.io/projected/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-kube-api-access-jmxzz\") pod \"payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.704622 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.704635 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.805466 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.805424 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r7x6\" (UniqueName: \"kubernetes.io/projected/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-kube-api-access-4r7x6\") pod \"payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.805659 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.805489 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxzz\" (UniqueName: \"kubernetes.io/projected/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-kube-api-access-jmxzz\") pod \"payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.805659 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.805512 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.805659 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.805533 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-plugins-config-volume\") pod \"payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.806236 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.806218 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.814576 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.814541 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxzz\" (UniqueName: \"kubernetes.io/projected/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-kube-api-access-jmxzz\") pod \"payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.906723 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.906680 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-plugins-config-volume\") pod \"payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.906882 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.906778 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r7x6\" (UniqueName: \"kubernetes.io/projected/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-kube-api-access-4r7x6\") pod \"payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.907552 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.907531 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-plugins-config-volume\") pod \"payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.912451 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.912430 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:17.914935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.914912 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r7x6\" (UniqueName: \"kubernetes.io/projected/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-kube-api-access-4r7x6\") pod \"payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:17.942222 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:17.942193 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:18.076457 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:18.076424 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89"] Jul 24 14:16:18.078542 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:16:18.078519 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa77123_529d_4439_8ca8_e6c3eec2a7eb.slice/crio-55c8eaa351de805cd390cfc3f657c179f6cf1d05a93362a9288f784c90da6f9b WatchSource:0}: Error finding container 55c8eaa351de805cd390cfc3f657c179f6cf1d05a93362a9288f784c90da6f9b: Status 404 returned error can't find the container with id 55c8eaa351de805cd390cfc3f657c179f6cf1d05a93362a9288f784c90da6f9b Jul 24 14:16:18.103186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:18.102942 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst"] Jul 24 14:16:18.106505 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:16:18.106474 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b27059c_13a9_4fe4_b5c2_40144e5aa11c.slice/crio-3b3c8cea7e89ee1ed3230cdbfd28a8d43168493f8bd0db9db81e43ec7132ba56 WatchSource:0}: Error finding container 3b3c8cea7e89ee1ed3230cdbfd28a8d43168493f8bd0db9db81e43ec7132ba56: Status 404 returned error can't find the container with id 3b3c8cea7e89ee1ed3230cdbfd28a8d43168493f8bd0db9db81e43ec7132ba56 Jul 24 14:16:18.110129 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:18.110104 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" event={"ID":"dfa77123-529d-4439-8ca8-e6c3eec2a7eb","Type":"ContainerStarted","Data":"55c8eaa351de805cd390cfc3f657c179f6cf1d05a93362a9288f784c90da6f9b"} Jul 24 14:16:19.117398 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.116967 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" event={"ID":"dfa77123-529d-4439-8ca8-e6c3eec2a7eb","Type":"ContainerStarted","Data":"7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8"} Jul 24 14:16:19.117398 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.117040 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:19.118732 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.118695 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" event={"ID":"2b27059c-13a9-4fe4-b5c2-40144e5aa11c","Type":"ContainerStarted","Data":"da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82"} Jul 24 14:16:19.118855 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.118745 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" event={"ID":"2b27059c-13a9-4fe4-b5c2-40144e5aa11c","Type":"ContainerStarted","Data":"3b3c8cea7e89ee1ed3230cdbfd28a8d43168493f8bd0db9db81e43ec7132ba56"} Jul 24 14:16:19.118855 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.118787 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:19.135563 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.135512 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" podStartSLOduration=2.135496727 podStartE2EDuration="2.135496727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:16:19.133130836 +0000 UTC m=+1865.928473597" watchObservedRunningTime="2026-07-24 14:16:19.135496727 +0000 UTC m=+1865.930839457" Jul 24 14:16:19.151817 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:19.151757 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" podStartSLOduration=2.15173856 podStartE2EDuration="2.15173856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:16:19.148546541 +0000 UTC m=+1865.943889282" watchObservedRunningTime="2026-07-24 14:16:19.15173856 +0000 UTC m=+1865.947081289" Jul 24 14:16:30.126548 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:30.126516 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:30.127037 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:30.126591 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:32.141192 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:32.141149 2555 patch_prober.go:28] interesting pod/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:16:32.141604 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:32.141229 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 14:16:33.191567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:33.191532 2555 generic.go:356] "Generic (PLEG): container finished" podID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerID="d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695" exitCode=0 Jul 24 14:16:33.192140 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:33.191591 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" event={"ID":"12f8d6ff-e518-4902-a677-a47ac24dd08e","Type":"ContainerDied","Data":"d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695"} Jul 24 14:16:33.192228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:33.192136 2555 scope.go:117] "RemoveContainer" containerID="d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695" Jul 24 14:16:33.193207 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:33.193179 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:33.193281 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:33.193247 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:33.193326 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:33.193297 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:34.202324 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:34.202231 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" event={"ID":"12f8d6ff-e518-4902-a677-a47ac24dd08e","Type":"ContainerStarted","Data":"1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330"} Jul 24 14:16:34.931422 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:34.931385 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:34.931775 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:34.931748 2555 patch_prober.go:28] interesting pod/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:16:34.931833 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:34.931811 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:16:35.931226 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:35.931187 2555 patch_prober.go:28] interesting pod/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:16:35.931643 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:35.931246 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:16:36.932185 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:36.932142 2555 patch_prober.go:28] interesting pod/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:16:36.932585 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:36.932216 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:16:37.931652 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:37.931617 2555 patch_prober.go:28] interesting pod/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 14:16:37.931839 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:37.931675 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 14:16:38.935662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:38.935625 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:38.936155 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:38.936129 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:38.936975 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:38.936956 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:16:52.831993 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:52.831935 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:52.856699 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:52.856664 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst"] Jul 24 14:16:52.856998 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:52.856953 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" podUID="2b27059c-13a9-4fe4-b5c2-40144e5aa11c" containerName="payload-processing" containerID="cri-o://da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82" gracePeriod=30 Jul 24 14:16:52.864143 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:52.864110 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89"] Jul 24 14:16:52.864402 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:52.864354 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" podUID="dfa77123-529d-4439-8ca8-e6c3eec2a7eb" containerName="payload-pre-processing" containerID="cri-o://7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8" gracePeriod=30 Jul 24 14:16:52.945399 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:52.945370 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:53.048192 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.046785 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:53.148014 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.147974 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:53.151041 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.151018 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:53.253098 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253061 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-plugins-config-volume\") pod \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " Jul 24 14:16:53.253303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253119 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-plugins-config-volume\") pod \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " Jul 24 14:16:53.253303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253187 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmxzz\" (UniqueName: \"kubernetes.io/projected/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-kube-api-access-jmxzz\") pod \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\" (UID: \"dfa77123-529d-4439-8ca8-e6c3eec2a7eb\") " Jul 24 14:16:53.253303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253228 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r7x6\" (UniqueName: \"kubernetes.io/projected/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-kube-api-access-4r7x6\") pod \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\" (UID: \"2b27059c-13a9-4fe4-b5c2-40144e5aa11c\") " Jul 24 14:16:53.253493 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253458 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dfa77123-529d-4439-8ca8-e6c3eec2a7eb" (UID: "dfa77123-529d-4439-8ca8-e6c3eec2a7eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:16:53.253573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253549 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b27059c-13a9-4fe4-b5c2-40144e5aa11c" (UID: "2b27059c-13a9-4fe4-b5c2-40144e5aa11c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:16:53.253622 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.253569 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:16:53.255662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.255630 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-kube-api-access-4r7x6" (OuterVolumeSpecName: "kube-api-access-4r7x6") pod "2b27059c-13a9-4fe4-b5c2-40144e5aa11c" (UID: "2b27059c-13a9-4fe4-b5c2-40144e5aa11c"). InnerVolumeSpecName "kube-api-access-4r7x6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:16:53.255662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.255631 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-kube-api-access-jmxzz" (OuterVolumeSpecName: "kube-api-access-jmxzz") pod "dfa77123-529d-4439-8ca8-e6c3eec2a7eb" (UID: "dfa77123-529d-4439-8ca8-e6c3eec2a7eb"). InnerVolumeSpecName "kube-api-access-jmxzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:16:53.300331 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.300283 2555 generic.go:356] "Generic (PLEG): container finished" podID="dfa77123-529d-4439-8ca8-e6c3eec2a7eb" containerID="7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8" exitCode=0 Jul 24 14:16:53.300503 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.300442 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" event={"ID":"dfa77123-529d-4439-8ca8-e6c3eec2a7eb","Type":"ContainerDied","Data":"7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8"} Jul 24 14:16:53.300586 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.300534 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" event={"ID":"dfa77123-529d-4439-8ca8-e6c3eec2a7eb","Type":"ContainerDied","Data":"55c8eaa351de805cd390cfc3f657c179f6cf1d05a93362a9288f784c90da6f9b"} Jul 24 14:16:53.300586 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.300574 2555 scope.go:117] "RemoveContainer" containerID="7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8" Jul 24 14:16:53.301120 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.300669 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89" Jul 24 14:16:53.307772 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.307727 2555 generic.go:356] "Generic (PLEG): container finished" podID="2b27059c-13a9-4fe4-b5c2-40144e5aa11c" containerID="da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82" exitCode=0 Jul 24 14:16:53.307881 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.307820 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" event={"ID":"2b27059c-13a9-4fe4-b5c2-40144e5aa11c","Type":"ContainerDied","Data":"da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82"} Jul 24 14:16:53.307961 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.307881 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" event={"ID":"2b27059c-13a9-4fe4-b5c2-40144e5aa11c","Type":"ContainerDied","Data":"3b3c8cea7e89ee1ed3230cdbfd28a8d43168493f8bd0db9db81e43ec7132ba56"} Jul 24 14:16:53.308048 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.308006 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst" Jul 24 14:16:53.316820 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.316796 2555 scope.go:117] "RemoveContainer" containerID="7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8" Jul 24 14:16:53.317234 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:16:53.317203 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8\": container with ID starting with 7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8 not found: ID does not exist" containerID="7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8" Jul 24 14:16:53.317396 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.317244 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8"} err="failed to get container status \"7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8\": rpc error: code = NotFound desc = could not find container \"7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8\": container with ID starting with 7cb6fe5c6a487a9025d1e44a67c49deee5499fb9d7aecc63c769e8a1ca4066a8 not found: ID does not exist" Jul 24 14:16:53.317396 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.317271 2555 scope.go:117] "RemoveContainer" containerID="da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82" Jul 24 14:16:53.326588 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.326568 2555 scope.go:117] "RemoveContainer" containerID="da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82" Jul 24 14:16:53.326953 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:16:53.326932 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82\": container with ID starting with da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82 not found: ID does not exist" containerID="da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82" Jul 24 14:16:53.327085 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.326964 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82"} err="failed to get container status \"da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82\": rpc error: code = NotFound desc = could not find container \"da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82\": container with ID starting with da1d0094ec7592bb95bb01c95d162dbb1a8cd873e699bb01c3bc21e9b0f82a82 not found: ID does not exist" Jul 24 14:16:53.328047 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.328022 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89"] Jul 24 14:16:53.333801 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.333770 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-600fea54-8c856b68-9sw89"] Jul 24 14:16:53.345217 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.345183 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst"] Jul 24 14:16:53.351199 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.351174 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-600fea54-7f554c49fd-kcfst"] Jul 24 14:16:53.354595 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.354577 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmxzz\" (UniqueName: \"kubernetes.io/projected/dfa77123-529d-4439-8ca8-e6c3eec2a7eb-kube-api-access-jmxzz\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:16:53.354660 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.354597 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r7x6\" (UniqueName: \"kubernetes.io/projected/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-kube-api-access-4r7x6\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:16:53.354660 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.354609 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b27059c-13a9-4fe4-b5c2-40144e5aa11c-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:16:53.750481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.750445 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b27059c-13a9-4fe4-b5c2-40144e5aa11c" path="/var/lib/kubelet/pods/2b27059c-13a9-4fe4-b5c2-40144e5aa11c/volumes" Jul 24 14:16:53.750817 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:53.750804 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfa77123-529d-4439-8ca8-e6c3eec2a7eb" path="/var/lib/kubelet/pods/dfa77123-529d-4439-8ca8-e6c3eec2a7eb/volumes" Jul 24 14:16:57.491160 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:57.490711 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv"] Jul 24 14:16:57.491160 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:57.490975 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" containerID="cri-o://1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330" gracePeriod=30 Jul 24 14:16:57.623711 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:57.623664 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:57.720951 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:57.720912 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:57.821045 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:57.820976 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:58.028518 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.028483 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:58.133321 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.133233 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:58.588283 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588245 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4"] Jul 24 14:16:58.588755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588641 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa77123-529d-4439-8ca8-e6c3eec2a7eb" containerName="payload-pre-processing" Jul 24 14:16:58.588755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588653 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa77123-529d-4439-8ca8-e6c3eec2a7eb" containerName="payload-pre-processing" Jul 24 14:16:58.588755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588666 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b27059c-13a9-4fe4-b5c2-40144e5aa11c" containerName="payload-processing" Jul 24 14:16:58.588755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588671 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b27059c-13a9-4fe4-b5c2-40144e5aa11c" containerName="payload-processing" Jul 24 14:16:58.588755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588738 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b27059c-13a9-4fe4-b5c2-40144e5aa11c" containerName="payload-processing" Jul 24 14:16:58.588755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.588749 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa77123-529d-4439-8ca8-e6c3eec2a7eb" containerName="payload-pre-processing" Jul 24 14:16:58.589742 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.589716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:58.591996 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.591970 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.596306 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.596288 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a7e2221a-gw-openshift-default-dockercfg-nsxfr\"" Jul 24 14:16:58.605005 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.604962 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-socket\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605209 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605182 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-credential-socket\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605318 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605227 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-data\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605318 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605276 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-certs\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605318 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605298 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-token\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605477 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605348 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tk6c\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-kube-api-access-6tk6c\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605477 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605376 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istiod-ca-cert\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605477 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605412 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-envoy\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.605477 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.605450 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-podinfo\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.607697 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.607675 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4"] Jul 24 14:16:58.668821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.668779 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:58.706487 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706450 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-socket\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.706682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706497 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-credential-socket\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.706682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706528 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-data\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.706682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706564 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-certs\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.706682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-token\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.706682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706624 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tk6c\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-kube-api-access-6tk6c\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.706682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706655 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istiod-ca-cert\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707028 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706687 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-envoy\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707028 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-podinfo\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707028 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.706926 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-socket\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707028 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.707016 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-certs\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707242 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.707060 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-credential-socket\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707439 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.707413 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-data\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.707591 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.707523 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istiod-ca-cert\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.709265 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.709236 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-envoy\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.709527 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.709511 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-podinfo\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.715744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.715719 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-token\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.715857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.715774 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tk6c\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-kube-api-access-6tk6c\") pod \"e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:58.874749 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.874666 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:16:58.902768 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:58.902733 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:16:59.050669 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.050637 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4"] Jul 24 14:16:59.053738 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:16:59.053715 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e29f99c_f0f1_4e89_8213_8db00b0cdc1f.slice/crio-e40e403465a0d2f763d8289ee57b3b9ce37da742f150584cf54d167d38fe622b WatchSource:0}: Error finding container e40e403465a0d2f763d8289ee57b3b9ce37da742f150584cf54d167d38fe622b: Status 404 returned error can't find the container with id e40e403465a0d2f763d8289ee57b3b9ce37da742f150584cf54d167d38fe622b Jul 24 14:16:59.055931 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.055897 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:59.056061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.055971 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:59.056061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.056040 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:16:59.343962 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.343915 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" event={"ID":"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f","Type":"ContainerStarted","Data":"1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065"} Jul 24 14:16:59.344185 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.343992 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" event={"ID":"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f","Type":"ContainerStarted","Data":"e40e403465a0d2f763d8289ee57b3b9ce37da742f150584cf54d167d38fe622b"} Jul 24 14:16:59.369702 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.369644 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podStartSLOduration=1.369626769 podStartE2EDuration="1.369626769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:16:59.367339427 +0000 UTC m=+1906.162682157" watchObservedRunningTime="2026-07-24 14:16:59.369626769 +0000 UTC m=+1906.164969570" Jul 24 14:16:59.903623 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:16:59.903589 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:00.854280 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:00.854245 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:00.909048 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:00.909006 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:00.989121 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:00.989082 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:01.128237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.128142 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:01.355395 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.355357 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:01.356504 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.356484 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:01.500736 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.500650 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:01.591110 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.591079 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:01.593155 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.593127 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm"] Jul 24 14:17:01.602376 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.602354 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.605713 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.605685 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-a7e2221a-dockercfg-6j2mx\"" Jul 24 14:17:01.605857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.605739 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-a7e2221a\"" Jul 24 14:17:01.607302 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.607275 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm"] Jul 24 14:17:01.609409 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.609385 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8"] Jul 24 14:17:01.619287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.619257 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.627072 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.626869 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8"] Jul 24 14:17:01.636958 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.636897 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxc48\" (UniqueName: \"kubernetes.io/projected/5c1dae12-237e-4bab-9316-2f8184b2f5a1-kube-api-access-wxc48\") pod \"payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.637428 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.637394 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a938ed18-1527-477a-9503-dffd1c71f29d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.637585 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.637566 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1dae12-237e-4bab-9316-2f8184b2f5a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.637732 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.637716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v82fk\" (UniqueName: \"kubernetes.io/projected/a938ed18-1527-477a-9503-dffd1c71f29d-kube-api-access-v82fk\") pod \"payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.738799 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.738760 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a938ed18-1527-477a-9503-dffd1c71f29d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.738799 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.738797 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1dae12-237e-4bab-9316-2f8184b2f5a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.739044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.738820 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v82fk\" (UniqueName: \"kubernetes.io/projected/a938ed18-1527-477a-9503-dffd1c71f29d-kube-api-access-v82fk\") pod \"payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.739044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.738851 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxc48\" (UniqueName: \"kubernetes.io/projected/5c1dae12-237e-4bab-9316-2f8184b2f5a1-kube-api-access-wxc48\") pod \"payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.739778 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.739753 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1dae12-237e-4bab-9316-2f8184b2f5a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.739831 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.739753 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a938ed18-1527-477a-9503-dffd1c71f29d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.748389 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.748302 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v82fk\" (UniqueName: \"kubernetes.io/projected/a938ed18-1527-477a-9503-dffd1c71f29d-kube-api-access-v82fk\") pod \"payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.748562 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.748448 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxc48\" (UniqueName: \"kubernetes.io/projected/5c1dae12-237e-4bab-9316-2f8184b2f5a1-kube-api-access-wxc48\") pod \"payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:01.915132 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.915101 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:01.932842 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:01.932809 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:02.071604 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.071551 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm"] Jul 24 14:17:02.073132 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:17:02.073106 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda938ed18_1527_477a_9503_dffd1c71f29d.slice/crio-470b5c393836a768f910026a55cde416eed1e8e16e3c8475ba9115ee34383210 WatchSource:0}: Error finding container 470b5c393836a768f910026a55cde416eed1e8e16e3c8475ba9115ee34383210: Status 404 returned error can't find the container with id 470b5c393836a768f910026a55cde416eed1e8e16e3c8475ba9115ee34383210 Jul 24 14:17:02.316628 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.316591 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8"] Jul 24 14:17:02.318490 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:17:02.318464 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1dae12_237e_4bab_9316_2f8184b2f5a1.slice/crio-c8f17da48fbff0e2a3a5bed51bc7d9170b47cfff0e4fad4e66c8c1b8a471c540 WatchSource:0}: Error finding container c8f17da48fbff0e2a3a5bed51bc7d9170b47cfff0e4fad4e66c8c1b8a471c540: Status 404 returned error can't find the container with id c8f17da48fbff0e2a3a5bed51bc7d9170b47cfff0e4fad4e66c8c1b8a471c540 Jul 24 14:17:02.362811 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.362774 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" event={"ID":"a938ed18-1527-477a-9503-dffd1c71f29d","Type":"ContainerStarted","Data":"02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9"} Jul 24 14:17:02.362938 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.362850 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" event={"ID":"a938ed18-1527-477a-9503-dffd1c71f29d","Type":"ContainerStarted","Data":"470b5c393836a768f910026a55cde416eed1e8e16e3c8475ba9115ee34383210"} Jul 24 14:17:02.362938 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.362897 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:02.364266 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.364227 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" event={"ID":"5c1dae12-237e-4bab-9316-2f8184b2f5a1","Type":"ContainerStarted","Data":"c8f17da48fbff0e2a3a5bed51bc7d9170b47cfff0e4fad4e66c8c1b8a471c540"} Jul 24 14:17:02.379854 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.379799 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" podStartSLOduration=1.379776191 podStartE2EDuration="1.379776191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:17:02.37807634 +0000 UTC m=+1909.173419071" watchObservedRunningTime="2026-07-24 14:17:02.379776191 +0000 UTC m=+1909.175118921" Jul 24 14:17:02.746906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.746877 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:17:02.850394 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850354 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-socket\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850394 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850400 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-envoy\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850423 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-data\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850449 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-podinfo\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850483 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj2p8\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-kube-api-access-sj2p8\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850556 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8d6ff-e518-4902-a677-a47ac24dd08e-istiod-ca-cert\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850610 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-certs\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850650 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850634 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-token\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850953 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850657 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-credential-socket\") pod \"12f8d6ff-e518-4902-a677-a47ac24dd08e\" (UID: \"12f8d6ff-e518-4902-a677-a47ac24dd08e\") " Jul 24 14:17:02.850953 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850740 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:02.851117 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.850956 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:02.851117 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.851084 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.851117 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.851104 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.851397 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.851376 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:02.851501 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.851473 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f8d6ff-e518-4902-a677-a47ac24dd08e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:17:02.851616 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.851591 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-data" (OuterVolumeSpecName: "istio-data") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:02.852945 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.852918 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:02.852945 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.852927 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-kube-api-access-sj2p8" (OuterVolumeSpecName: "kube-api-access-sj2p8") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "kube-api-access-sj2p8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:17:02.853211 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.853115 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:17:02.853467 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.853441 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-token" (OuterVolumeSpecName: "istio-token") pod "12f8d6ff-e518-4902-a677-a47ac24dd08e" (UID: "12f8d6ff-e518-4902-a677-a47ac24dd08e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:17:02.952476 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952438 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.952476 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952469 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.952476 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952477 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.952476 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952486 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj2p8\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-kube-api-access-sj2p8\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.952968 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952495 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8d6ff-e518-4902-a677-a47ac24dd08e-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.952968 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952503 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8d6ff-e518-4902-a677-a47ac24dd08e-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:02.952968 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:02.952511 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8d6ff-e518-4902-a677-a47ac24dd08e-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:03.371782 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.371746 2555 generic.go:356] "Generic (PLEG): container finished" podID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerID="1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330" exitCode=0 Jul 24 14:17:03.372068 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.371804 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" Jul 24 14:17:03.372068 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.371827 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" event={"ID":"12f8d6ff-e518-4902-a677-a47ac24dd08e","Type":"ContainerDied","Data":"1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330"} Jul 24 14:17:03.372068 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.371888 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv" event={"ID":"12f8d6ff-e518-4902-a677-a47ac24dd08e","Type":"ContainerDied","Data":"ee908e1137f74a7b6871437ac747ef19eeafbcd1fcfe12a2b498be0a139c8f3e"} Jul 24 14:17:03.372068 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.371915 2555 scope.go:117] "RemoveContainer" containerID="1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330" Jul 24 14:17:03.373701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.373634 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" event={"ID":"5c1dae12-237e-4bab-9316-2f8184b2f5a1","Type":"ContainerStarted","Data":"fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101"} Jul 24 14:17:03.373891 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.373867 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:03.381969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.381950 2555 scope.go:117] "RemoveContainer" containerID="d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695" Jul 24 14:17:03.392145 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.392117 2555 scope.go:117] "RemoveContainer" containerID="1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330" Jul 24 14:17:03.392487 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:17:03.392449 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330\": container with ID starting with 1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330 not found: ID does not exist" containerID="1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330" Jul 24 14:17:03.392590 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.392516 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330"} err="failed to get container status \"1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330\": rpc error: code = NotFound desc = could not find container \"1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330\": container with ID starting with 1ecfca2edc5bad1c202bdc57095480c8c27cc12ed8c4bf084bb3a8414c4b6330 not found: ID does not exist" Jul 24 14:17:03.392590 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.392544 2555 scope.go:117] "RemoveContainer" containerID="d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695" Jul 24 14:17:03.392895 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:17:03.392863 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695\": container with ID starting with d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695 not found: ID does not exist" containerID="d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695" Jul 24 14:17:03.392961 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.392904 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695"} err="failed to get container status \"d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695\": rpc error: code = NotFound desc = could not find container \"d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695\": container with ID starting with d215390761913fc6052727f3099f2019e772c6164aed83d1acbb6f4504071695 not found: ID does not exist" Jul 24 14:17:03.396686 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.396643 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" podStartSLOduration=2.396628117 podStartE2EDuration="2.396628117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:17:03.395283489 +0000 UTC m=+1910.190626221" watchObservedRunningTime="2026-07-24 14:17:03.396628117 +0000 UTC m=+1910.191970848" Jul 24 14:17:03.412463 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.412427 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv"] Jul 24 14:17:03.416681 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.416657 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-600fea54-openshift-default-6d5d895678-mh6qv"] Jul 24 14:17:03.752646 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:03.752569 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" path="/var/lib/kubelet/pods/12f8d6ff-e518-4902-a677-a47ac24dd08e/volumes" Jul 24 14:17:13.376045 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:13.375978 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:14.383588 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:14.383552 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:16.913219 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:16.912868 2555 patch_prober.go:28] interesting pod/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:42188->10.133.0.51:15021: read: connection reset by peer" start-of-body= Jul 24 14:17:16.913219 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:16.912955 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:42188->10.133.0.51:15021: read: connection reset by peer" Jul 24 14:17:17.451718 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:17.451682 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerID="1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065" exitCode=0 Jul 24 14:17:17.451899 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:17.451765 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" event={"ID":"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f","Type":"ContainerDied","Data":"1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065"} Jul 24 14:17:17.452231 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:17.452217 2555 scope.go:117] "RemoveContainer" containerID="1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065" Jul 24 14:17:17.453222 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:17.453191 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:17:17.453339 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:17.453256 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:17:17.453339 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:17.453288 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:17:18.458444 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:18.458412 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" event={"ID":"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f","Type":"ContainerStarted","Data":"3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db"} Jul 24 14:17:18.903264 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:18.903219 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:18.903903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:18.903876 2555 patch_prober.go:28] interesting pod/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:17:18.904151 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:18.904124 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:17:19.904020 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:19.903942 2555 patch_prober.go:28] interesting pod/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:17:19.904656 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:19.904623 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:17:20.903472 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:20.903429 2555 patch_prober.go:28] interesting pod/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:17:20.903801 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:20.903769 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:17:21.907168 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:21.907130 2555 patch_prober.go:28] interesting pod/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 14:17:21.907754 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:21.907721 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 14:17:22.908191 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:22.908162 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:22.908589 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:22.908457 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:22.909447 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:22.909383 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:36.498635 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.498592 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:36.514473 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.514437 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8"] Jul 24 14:17:36.514764 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.514705 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" podUID="5c1dae12-237e-4bab-9316-2f8184b2f5a1" containerName="payload-processing" containerID="cri-o://fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101" gracePeriod=30 Jul 24 14:17:36.525624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.523247 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm"] Jul 24 14:17:36.525624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.523487 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" podUID="a938ed18-1527-477a-9503-dffd1c71f29d" containerName="payload-pre-processing" containerID="cri-o://02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9" gracePeriod=30 Jul 24 14:17:36.598762 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.598731 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:36.777844 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.777817 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:36.812535 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.812509 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:36.880682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.880649 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxc48\" (UniqueName: \"kubernetes.io/projected/5c1dae12-237e-4bab-9316-2f8184b2f5a1-kube-api-access-wxc48\") pod \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " Jul 24 14:17:36.880682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.880685 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1dae12-237e-4bab-9316-2f8184b2f5a1-plugins-config-volume\") pod \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\" (UID: \"5c1dae12-237e-4bab-9316-2f8184b2f5a1\") " Jul 24 14:17:36.881179 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.881153 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c1dae12-237e-4bab-9316-2f8184b2f5a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c1dae12-237e-4bab-9316-2f8184b2f5a1" (UID: "5c1dae12-237e-4bab-9316-2f8184b2f5a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:17:36.883100 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.883080 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1dae12-237e-4bab-9316-2f8184b2f5a1-kube-api-access-wxc48" (OuterVolumeSpecName: "kube-api-access-wxc48") pod "5c1dae12-237e-4bab-9316-2f8184b2f5a1" (UID: "5c1dae12-237e-4bab-9316-2f8184b2f5a1"). InnerVolumeSpecName "kube-api-access-wxc48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:17:36.982089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.982053 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v82fk\" (UniqueName: \"kubernetes.io/projected/a938ed18-1527-477a-9503-dffd1c71f29d-kube-api-access-v82fk\") pod \"a938ed18-1527-477a-9503-dffd1c71f29d\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " Jul 24 14:17:36.982291 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.982145 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a938ed18-1527-477a-9503-dffd1c71f29d-plugins-config-volume\") pod \"a938ed18-1527-477a-9503-dffd1c71f29d\" (UID: \"a938ed18-1527-477a-9503-dffd1c71f29d\") " Jul 24 14:17:36.982438 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.982411 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxc48\" (UniqueName: \"kubernetes.io/projected/5c1dae12-237e-4bab-9316-2f8184b2f5a1-kube-api-access-wxc48\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:36.982438 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.982431 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c1dae12-237e-4bab-9316-2f8184b2f5a1-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:36.982577 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.982494 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a938ed18-1527-477a-9503-dffd1c71f29d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a938ed18-1527-477a-9503-dffd1c71f29d" (UID: "a938ed18-1527-477a-9503-dffd1c71f29d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:17:36.984436 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:36.984410 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a938ed18-1527-477a-9503-dffd1c71f29d-kube-api-access-v82fk" (OuterVolumeSpecName: "kube-api-access-v82fk") pod "a938ed18-1527-477a-9503-dffd1c71f29d" (UID: "a938ed18-1527-477a-9503-dffd1c71f29d"). InnerVolumeSpecName "kube-api-access-v82fk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:17:37.083335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.083296 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v82fk\" (UniqueName: \"kubernetes.io/projected/a938ed18-1527-477a-9503-dffd1c71f29d-kube-api-access-v82fk\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:37.083335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.083331 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a938ed18-1527-477a-9503-dffd1c71f29d-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:37.555303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.555268 2555 generic.go:356] "Generic (PLEG): container finished" podID="5c1dae12-237e-4bab-9316-2f8184b2f5a1" containerID="fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101" exitCode=0 Jul 24 14:17:37.555750 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.555317 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" Jul 24 14:17:37.555750 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.555351 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" event={"ID":"5c1dae12-237e-4bab-9316-2f8184b2f5a1","Type":"ContainerDied","Data":"fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101"} Jul 24 14:17:37.555750 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.555405 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8" event={"ID":"5c1dae12-237e-4bab-9316-2f8184b2f5a1","Type":"ContainerDied","Data":"c8f17da48fbff0e2a3a5bed51bc7d9170b47cfff0e4fad4e66c8c1b8a471c540"} Jul 24 14:17:37.555750 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.555434 2555 scope.go:117] "RemoveContainer" containerID="fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101" Jul 24 14:17:37.556802 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.556782 2555 generic.go:356] "Generic (PLEG): container finished" podID="a938ed18-1527-477a-9503-dffd1c71f29d" containerID="02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9" exitCode=0 Jul 24 14:17:37.556881 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.556827 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" Jul 24 14:17:37.556881 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.556845 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" event={"ID":"a938ed18-1527-477a-9503-dffd1c71f29d","Type":"ContainerDied","Data":"02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9"} Jul 24 14:17:37.556966 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.556902 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm" event={"ID":"a938ed18-1527-477a-9503-dffd1c71f29d","Type":"ContainerDied","Data":"470b5c393836a768f910026a55cde416eed1e8e16e3c8475ba9115ee34383210"} Jul 24 14:17:37.566025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.565966 2555 scope.go:117] "RemoveContainer" containerID="fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101" Jul 24 14:17:37.566323 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:17:37.566302 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101\": container with ID starting with fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101 not found: ID does not exist" containerID="fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101" Jul 24 14:17:37.566418 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.566335 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101"} err="failed to get container status \"fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101\": rpc error: code = NotFound desc = could not find container \"fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101\": container with ID starting with fcd5c8441ca72d07e16164cc5363b35b7fcb4cc8b62b206604b34137548d8101 not found: ID does not exist" Jul 24 14:17:37.566418 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.566357 2555 scope.go:117] "RemoveContainer" containerID="02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9" Jul 24 14:17:37.574826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.574807 2555 scope.go:117] "RemoveContainer" containerID="02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9" Jul 24 14:17:37.575190 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:17:37.575168 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9\": container with ID starting with 02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9 not found: ID does not exist" containerID="02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9" Jul 24 14:17:37.575283 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.575201 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9"} err="failed to get container status \"02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9\": rpc error: code = NotFound desc = could not find container \"02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9\": container with ID starting with 02aed66cfbee12d8b509b9018d76a7214e10926a67a8501cfdade4a0e51494b9 not found: ID does not exist" Jul 24 14:17:37.580976 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.580951 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm"] Jul 24 14:17:37.585369 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.585344 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-a7e2221a-6bfd8f65d4-89vtm"] Jul 24 14:17:37.598883 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.598851 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8"] Jul 24 14:17:37.602398 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.602373 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-a7e2221a-54ddb77b49-z7tl8"] Jul 24 14:17:37.753891 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.753838 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c1dae12-237e-4bab-9316-2f8184b2f5a1" path="/var/lib/kubelet/pods/5c1dae12-237e-4bab-9316-2f8184b2f5a1/volumes" Jul 24 14:17:37.754250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:37.754236 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a938ed18-1527-477a-9503-dffd1c71f29d" path="/var/lib/kubelet/pods/a938ed18-1527-477a-9503-dffd1c71f29d/volumes" Jul 24 14:17:40.951391 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:40.951351 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:41.083434 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.083398 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4"] Jul 24 14:17:41.083690 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.083658 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" containerID="cri-o://3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db" gracePeriod=30 Jul 24 14:17:41.109238 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.109207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:41.341411 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.341373 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:41.567721 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.567677 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:41.854058 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854021 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm"] Jul 24 14:17:41.854465 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854444 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" Jul 24 14:17:41.854541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854470 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" Jul 24 14:17:41.854541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854484 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" Jul 24 14:17:41.854541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854494 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" Jul 24 14:17:41.854541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854508 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a938ed18-1527-477a-9503-dffd1c71f29d" containerName="payload-pre-processing" Jul 24 14:17:41.854541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854517 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a938ed18-1527-477a-9503-dffd1c71f29d" containerName="payload-pre-processing" Jul 24 14:17:41.854701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854556 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c1dae12-237e-4bab-9316-2f8184b2f5a1" containerName="payload-processing" Jul 24 14:17:41.854701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854564 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c1dae12-237e-4bab-9316-2f8184b2f5a1" containerName="payload-processing" Jul 24 14:17:41.854701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854651 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c1dae12-237e-4bab-9316-2f8184b2f5a1" containerName="payload-processing" Jul 24 14:17:41.854701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854665 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" Jul 24 14:17:41.854701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.854675 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a938ed18-1527-477a-9503-dffd1c71f29d" containerName="payload-pre-processing" Jul 24 14:17:41.858358 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.858337 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:41.861086 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.861062 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1f2fc6f2-openshift-default-dockercfg-4xfz2\"" Jul 24 14:17:41.892608 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.891699 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm"] Jul 24 14:17:41.892608 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:41.891712 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:42.026198 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026157 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-data\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026198 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026196 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-envoy\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026240 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-credential-socket\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnvkr\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-kube-api-access-qnvkr\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026397 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-podinfo\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026476 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-certs\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026525 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51df5696-7d52-4cfd-a5f7-016663b4dc94-istiod-ca-cert\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026565 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-token\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.026813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.026598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-socket\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.084803 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.084772 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:42.128262 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-socket\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128262 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128253 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-data\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128450 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128281 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-envoy\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128450 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128331 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-credential-socket\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128450 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128361 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnvkr\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-kube-api-access-qnvkr\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128450 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128390 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-podinfo\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128461 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-certs\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128497 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51df5696-7d52-4cfd-a5f7-016663b4dc94-istiod-ca-cert\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128543 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-token\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128792 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128733 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-data\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128849 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128786 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-credential-socket\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128849 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128804 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-certs\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.128957 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.128863 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-socket\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.129289 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.129270 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51df5696-7d52-4cfd-a5f7-016663b4dc94-istiod-ca-cert\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.130930 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.130908 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-envoy\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.131192 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.131172 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-podinfo\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.138104 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.138072 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnvkr\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-kube-api-access-qnvkr\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.139573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.139544 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-token\") pod \"e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.182277 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.182239 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:42.327385 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.327359 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm"] Jul 24 14:17:42.329269 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:17:42.329236 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51df5696_7d52_4cfd_a5f7_016663b4dc94.slice/crio-54ca168a4c26d58068ebe305888f96d660c59a750b6ab32eda7062ce8e472005 WatchSource:0}: Error finding container 54ca168a4c26d58068ebe305888f96d660c59a750b6ab32eda7062ce8e472005: Status 404 returned error can't find the container with id 54ca168a4c26d58068ebe305888f96d660c59a750b6ab32eda7062ce8e472005 Jul 24 14:17:42.331423 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.331390 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:17:42.331505 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.331461 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:17:42.331505 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.331492 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:17:42.587580 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.587542 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" event={"ID":"51df5696-7d52-4cfd-a5f7-016663b4dc94","Type":"ContainerStarted","Data":"6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a"} Jul 24 14:17:42.587753 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.587612 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" event={"ID":"51df5696-7d52-4cfd-a5f7-016663b4dc94","Type":"ContainerStarted","Data":"54ca168a4c26d58068ebe305888f96d660c59a750b6ab32eda7062ce8e472005"} Jul 24 14:17:42.616109 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:42.616019 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podStartSLOduration=1.615975362 podStartE2EDuration="1.615975362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:17:42.615667939 +0000 UTC m=+1949.411010672" watchObservedRunningTime="2026-07-24 14:17:42.615975362 +0000 UTC m=+1949.411318093" Jul 24 14:17:43.182949 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:43.182904 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:43.188128 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:43.188099 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:43.593128 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:43.593097 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:43.594235 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:43.594210 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:17:43.945153 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:43.945064 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:44.025735 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.025700 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:44.094373 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.094332 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:44.239115 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.239023 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:44.532698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.532606 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:44.623684 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.623647 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:17:44.630497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.630466 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z"] Jul 24 14:17:44.630996 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.630963 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="12f8d6ff-e518-4902-a677-a47ac24dd08e" containerName="istio-proxy" Jul 24 14:17:44.633375 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.633352 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.636414 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.636396 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1f2fc6f2\"" Jul 24 14:17:44.651892 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.651863 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z"] Jul 24 14:17:44.655067 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.655036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp57b\" (UniqueName: \"kubernetes.io/projected/9630a2f5-785b-469a-9a24-af6e963d48e9-kube-api-access-pp57b\") pod \"payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.655197 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.655180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9630a2f5-785b-469a-9a24-af6e963d48e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.698848 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.698811 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz"] Jul 24 14:17:44.701261 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.701245 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.704677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.704657 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1f2fc6f2-dockercfg-tbw44\"" Jul 24 14:17:44.720759 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.720726 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz"] Jul 24 14:17:44.756086 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.756050 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp57b\" (UniqueName: \"kubernetes.io/projected/9630a2f5-785b-469a-9a24-af6e963d48e9-kube-api-access-pp57b\") pod \"payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.756294 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.756119 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9630a2f5-785b-469a-9a24-af6e963d48e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.756294 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.756150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsszs\" (UniqueName: \"kubernetes.io/projected/5461be99-43be-4dc0-b1f7-33cf2c6001b7-kube-api-access-hsszs\") pod \"payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.756294 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.756194 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461be99-43be-4dc0-b1f7-33cf2c6001b7-plugins-config-volume\") pod \"payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.756779 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.756759 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9630a2f5-785b-469a-9a24-af6e963d48e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.770049 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.770014 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp57b\" (UniqueName: \"kubernetes.io/projected/9630a2f5-785b-469a-9a24-af6e963d48e9-kube-api-access-pp57b\") pod \"payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:44.857114 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.857070 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsszs\" (UniqueName: \"kubernetes.io/projected/5461be99-43be-4dc0-b1f7-33cf2c6001b7-kube-api-access-hsszs\") pod \"payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.857291 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.857134 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461be99-43be-4dc0-b1f7-33cf2c6001b7-plugins-config-volume\") pod \"payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.857833 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.857813 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461be99-43be-4dc0-b1f7-33cf2c6001b7-plugins-config-volume\") pod \"payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.869202 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.869170 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsszs\" (UniqueName: \"kubernetes.io/projected/5461be99-43be-4dc0-b1f7-33cf2c6001b7-kube-api-access-hsszs\") pod \"payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:44.944758 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:44.944717 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:45.011291 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.011261 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:45.112290 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.110792 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z"] Jul 24 14:17:45.112904 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:17:45.112842 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9630a2f5_785b_469a_9a24_af6e963d48e9.slice/crio-f8eaae390eb5b65912b81bd64e253125eed8d25251966d8591d0a1800997beac WatchSource:0}: Error finding container f8eaae390eb5b65912b81bd64e253125eed8d25251966d8591d0a1800997beac: Status 404 returned error can't find the container with id f8eaae390eb5b65912b81bd64e253125eed8d25251966d8591d0a1800997beac Jul 24 14:17:45.173520 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.173489 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz"] Jul 24 14:17:45.176514 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:17:45.176490 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5461be99_43be_4dc0_b1f7_33cf2c6001b7.slice/crio-33852d5842e305e14bdf8613bd3e658e09ba64afdc824eeb7194023519c8adcd WatchSource:0}: Error finding container 33852d5842e305e14bdf8613bd3e658e09ba64afdc824eeb7194023519c8adcd: Status 404 returned error can't find the container with id 33852d5842e305e14bdf8613bd3e658e09ba64afdc824eeb7194023519c8adcd Jul 24 14:17:45.605175 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.605139 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" event={"ID":"5461be99-43be-4dc0-b1f7-33cf2c6001b7","Type":"ContainerStarted","Data":"28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e"} Jul 24 14:17:45.605639 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.605194 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" event={"ID":"5461be99-43be-4dc0-b1f7-33cf2c6001b7","Type":"ContainerStarted","Data":"33852d5842e305e14bdf8613bd3e658e09ba64afdc824eeb7194023519c8adcd"} Jul 24 14:17:45.605639 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.605223 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:45.606763 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.606737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" event={"ID":"9630a2f5-785b-469a-9a24-af6e963d48e9","Type":"ContainerStarted","Data":"de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89"} Jul 24 14:17:45.606879 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.606789 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" event={"ID":"9630a2f5-785b-469a-9a24-af6e963d48e9","Type":"ContainerStarted","Data":"f8eaae390eb5b65912b81bd64e253125eed8d25251966d8591d0a1800997beac"} Jul 24 14:17:45.624207 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.624154 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" podStartSLOduration=1.624139076 podStartE2EDuration="1.624139076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:17:45.621950113 +0000 UTC m=+1952.417292843" watchObservedRunningTime="2026-07-24 14:17:45.624139076 +0000 UTC m=+1952.419481802" Jul 24 14:17:45.640231 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:45.640178 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" podStartSLOduration=1.640161493 podStartE2EDuration="1.640161493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:17:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:17:45.637719448 +0000 UTC m=+1952.433062189" watchObservedRunningTime="2026-07-24 14:17:45.640161493 +0000 UTC m=+1952.435504271" Jul 24 14:17:46.350047 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.349958 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:46.370063 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370023 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tk6c\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-kube-api-access-6tk6c\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370075 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-socket\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370094 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-podinfo\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370123 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-envoy\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370155 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-certs\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370201 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-data\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370261 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-credential-socket\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370309 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-token\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370333 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istiod-ca-cert\") pod \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\" (UID: \"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f\") " Jul 24 14:17:46.370497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370414 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:46.370694 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370493 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:46.370694 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370669 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:46.370694 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370681 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.370824 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370701 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.370824 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.370726 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:17:46.371475 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.371417 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:46.372551 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.372529 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:17:46.372655 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.372581 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-kube-api-access-6tk6c" (OuterVolumeSpecName: "kube-api-access-6tk6c") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "kube-api-access-6tk6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:17:46.372655 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.372633 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:17:46.372846 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.372832 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" (UID: "4e29f99c-f0f1-4e89-8213-8db00b0cdc1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:17:46.471755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471723 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.471755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471750 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.471755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471761 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.472044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471771 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.472044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471779 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tk6c\" (UniqueName: \"kubernetes.io/projected/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-kube-api-access-6tk6c\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.472044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471787 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.472044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.471794 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:17:46.613643 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.613554 2555 generic.go:356] "Generic (PLEG): container finished" podID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerID="3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db" exitCode=0 Jul 24 14:17:46.613643 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.613603 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" Jul 24 14:17:46.614159 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.613636 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" event={"ID":"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f","Type":"ContainerDied","Data":"3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db"} Jul 24 14:17:46.614159 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.613700 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4" event={"ID":"4e29f99c-f0f1-4e89-8213-8db00b0cdc1f","Type":"ContainerDied","Data":"e40e403465a0d2f763d8289ee57b3b9ce37da742f150584cf54d167d38fe622b"} Jul 24 14:17:46.614159 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.613731 2555 scope.go:117] "RemoveContainer" containerID="3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db" Jul 24 14:17:46.614159 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.614026 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:46.624743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.624725 2555 scope.go:117] "RemoveContainer" containerID="1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065" Jul 24 14:17:46.633532 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.633505 2555 scope.go:117] "RemoveContainer" containerID="3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db" Jul 24 14:17:46.633853 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:17:46.633834 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db\": container with ID starting with 3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db not found: ID does not exist" containerID="3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db" Jul 24 14:17:46.633922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.633866 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db"} err="failed to get container status \"3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db\": rpc error: code = NotFound desc = could not find container \"3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db\": container with ID starting with 3d05f3106bb313bbb8b8785521791e66d062cdc255863affcabf7631d2e0b3db not found: ID does not exist" Jul 24 14:17:46.633922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.633887 2555 scope.go:117] "RemoveContainer" containerID="1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065" Jul 24 14:17:46.634128 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:17:46.634115 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065\": container with ID starting with 1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065 not found: ID does not exist" containerID="1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065" Jul 24 14:17:46.634181 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.634133 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065"} err="failed to get container status \"1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065\": rpc error: code = NotFound desc = could not find container \"1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065\": container with ID starting with 1168bf0cd4c4fe917b3dedaccccd951ee25d3b5b157afb1a04c883d1e2217065 not found: ID does not exist" Jul 24 14:17:46.639102 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.639077 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4"] Jul 24 14:17:46.643688 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:46.643664 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a7e2221a-gw-openshift-default-5658768d-wqwz4"] Jul 24 14:17:47.750146 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:47.750115 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" path="/var/lib/kubelet/pods/4e29f99c-f0f1-4e89-8213-8db00b0cdc1f/volumes" Jul 24 14:17:56.616593 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:56.616561 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:17:57.621961 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:57.621930 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:17:59.635207 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:59.635154 2555 patch_prober.go:28] interesting pod/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:17:59.635731 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:17:59.635254 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 14:18:00.688505 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:00.688469 2555 generic.go:356] "Generic (PLEG): container finished" podID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerID="6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a" exitCode=0 Jul 24 14:18:00.689071 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:00.688533 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" event={"ID":"51df5696-7d52-4cfd-a5f7-016663b4dc94","Type":"ContainerDied","Data":"6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a"} Jul 24 14:18:00.689071 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:00.688921 2555 scope.go:117] "RemoveContainer" containerID="6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a" Jul 24 14:18:00.690097 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:00.690067 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:00.690181 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:00.690134 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:00.690181 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:00.690166 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:01.695768 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:01.695737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" event={"ID":"51df5696-7d52-4cfd-a5f7-016663b4dc94","Type":"ContainerStarted","Data":"414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63"} Jul 24 14:18:02.183310 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:02.183183 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:18:02.183900 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:02.183871 2555 patch_prober.go:28] interesting pod/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:18:02.184156 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:02.184128 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:18:03.183171 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:03.183133 2555 patch_prober.go:28] interesting pod/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:18:03.183773 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:03.183742 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:18:04.186758 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:04.186514 2555 patch_prober.go:28] interesting pod/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:18:04.186758 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:04.186595 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:18:05.196562 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:05.196509 2555 patch_prober.go:28] interesting pod/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:18:05.197092 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:05.196599 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:18:06.183593 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:06.183550 2555 patch_prober.go:28] interesting pod/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 14:18:06.184362 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:06.183914 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 14:18:07.186204 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:07.186171 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:18:07.186648 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:07.186636 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:18:07.187300 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:07.187279 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:18:14.721003 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:14.720942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:20.314613 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.314567 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:20.342557 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.342521 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz"] Jul 24 14:18:20.342832 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.342799 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" podUID="5461be99-43be-4dc0-b1f7-33cf2c6001b7" containerName="payload-processing" containerID="cri-o://28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e" gracePeriod=30 Jul 24 14:18:20.353662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.353628 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z"] Jul 24 14:18:20.353903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.353872 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" podUID="9630a2f5-785b-469a-9a24-af6e963d48e9" containerName="payload-pre-processing" containerID="cri-o://de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89" gracePeriod=30 Jul 24 14:18:20.406249 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.406210 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:20.487586 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.487522 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:20.625449 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.625420 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:18:20.649837 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.649810 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:18:20.793438 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.793402 2555 generic.go:356] "Generic (PLEG): container finished" podID="5461be99-43be-4dc0-b1f7-33cf2c6001b7" containerID="28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e" exitCode=0 Jul 24 14:18:20.793438 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.793441 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" Jul 24 14:18:20.793687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.793485 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" event={"ID":"5461be99-43be-4dc0-b1f7-33cf2c6001b7","Type":"ContainerDied","Data":"28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e"} Jul 24 14:18:20.793687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.793540 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz" event={"ID":"5461be99-43be-4dc0-b1f7-33cf2c6001b7","Type":"ContainerDied","Data":"33852d5842e305e14bdf8613bd3e658e09ba64afdc824eeb7194023519c8adcd"} Jul 24 14:18:20.793687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.793564 2555 scope.go:117] "RemoveContainer" containerID="28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e" Jul 24 14:18:20.794874 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.794843 2555 generic.go:356] "Generic (PLEG): container finished" podID="9630a2f5-785b-469a-9a24-af6e963d48e9" containerID="de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89" exitCode=0 Jul 24 14:18:20.794977 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.794881 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" Jul 24 14:18:20.794977 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.794887 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" event={"ID":"9630a2f5-785b-469a-9a24-af6e963d48e9","Type":"ContainerDied","Data":"de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89"} Jul 24 14:18:20.794977 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.794923 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z" event={"ID":"9630a2f5-785b-469a-9a24-af6e963d48e9","Type":"ContainerDied","Data":"f8eaae390eb5b65912b81bd64e253125eed8d25251966d8591d0a1800997beac"} Jul 24 14:18:20.802410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.802393 2555 scope.go:117] "RemoveContainer" containerID="28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e" Jul 24 14:18:20.802655 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:18:20.802637 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e\": container with ID starting with 28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e not found: ID does not exist" containerID="28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e" Jul 24 14:18:20.802716 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.802665 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e"} err="failed to get container status \"28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e\": rpc error: code = NotFound desc = could not find container \"28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e\": container with ID starting with 28414016aee575dc45191f3cf0401375e5d6d5777467d4f6bd6fe02dfbc5be8e not found: ID does not exist" Jul 24 14:18:20.802716 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.802681 2555 scope.go:117] "RemoveContainer" containerID="de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89" Jul 24 14:18:20.804851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.804829 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461be99-43be-4dc0-b1f7-33cf2c6001b7-plugins-config-volume\") pod \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " Jul 24 14:18:20.804933 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.804860 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9630a2f5-785b-469a-9a24-af6e963d48e9-plugins-config-volume\") pod \"9630a2f5-785b-469a-9a24-af6e963d48e9\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " Jul 24 14:18:20.804933 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.804908 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp57b\" (UniqueName: \"kubernetes.io/projected/9630a2f5-785b-469a-9a24-af6e963d48e9-kube-api-access-pp57b\") pod \"9630a2f5-785b-469a-9a24-af6e963d48e9\" (UID: \"9630a2f5-785b-469a-9a24-af6e963d48e9\") " Jul 24 14:18:20.805034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.804977 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsszs\" (UniqueName: \"kubernetes.io/projected/5461be99-43be-4dc0-b1f7-33cf2c6001b7-kube-api-access-hsszs\") pod \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\" (UID: \"5461be99-43be-4dc0-b1f7-33cf2c6001b7\") " Jul 24 14:18:20.805273 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.805249 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5461be99-43be-4dc0-b1f7-33cf2c6001b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5461be99-43be-4dc0-b1f7-33cf2c6001b7" (UID: "5461be99-43be-4dc0-b1f7-33cf2c6001b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:18:20.805352 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.805310 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9630a2f5-785b-469a-9a24-af6e963d48e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9630a2f5-785b-469a-9a24-af6e963d48e9" (UID: "9630a2f5-785b-469a-9a24-af6e963d48e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:18:20.805456 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.805439 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5461be99-43be-4dc0-b1f7-33cf2c6001b7-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:20.805511 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.805464 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9630a2f5-785b-469a-9a24-af6e963d48e9-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:20.807253 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.807228 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5461be99-43be-4dc0-b1f7-33cf2c6001b7-kube-api-access-hsszs" (OuterVolumeSpecName: "kube-api-access-hsszs") pod "5461be99-43be-4dc0-b1f7-33cf2c6001b7" (UID: "5461be99-43be-4dc0-b1f7-33cf2c6001b7"). InnerVolumeSpecName "kube-api-access-hsszs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:18:20.807366 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.807272 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9630a2f5-785b-469a-9a24-af6e963d48e9-kube-api-access-pp57b" (OuterVolumeSpecName: "kube-api-access-pp57b") pod "9630a2f5-785b-469a-9a24-af6e963d48e9" (UID: "9630a2f5-785b-469a-9a24-af6e963d48e9"). InnerVolumeSpecName "kube-api-access-pp57b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:18:20.811338 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.811322 2555 scope.go:117] "RemoveContainer" containerID="de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89" Jul 24 14:18:20.811618 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:18:20.811599 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89\": container with ID starting with de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89 not found: ID does not exist" containerID="de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89" Jul 24 14:18:20.811669 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.811627 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89"} err="failed to get container status \"de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89\": rpc error: code = NotFound desc = could not find container \"de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89\": container with ID starting with de373171582c08edee062ac2b79094b18eb785b5031db0a056d6692c9bb82f89 not found: ID does not exist" Jul 24 14:18:20.906866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.906779 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp57b\" (UniqueName: \"kubernetes.io/projected/9630a2f5-785b-469a-9a24-af6e963d48e9-kube-api-access-pp57b\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:20.906866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:20.906809 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsszs\" (UniqueName: \"kubernetes.io/projected/5461be99-43be-4dc0-b1f7-33cf2c6001b7-kube-api-access-hsszs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:21.124140 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:21.124104 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z"] Jul 24 14:18:21.128457 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:21.128430 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1f2fc6f2-b466dcc56-5q82z"] Jul 24 14:18:21.142174 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:21.142147 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz"] Jul 24 14:18:21.148175 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:21.148147 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1f2fc6f2-5dd5849dbd-q4vcz"] Jul 24 14:18:21.748898 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:21.748819 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5461be99-43be-4dc0-b1f7-33cf2c6001b7" path="/var/lib/kubelet/pods/5461be99-43be-4dc0-b1f7-33cf2c6001b7/volumes" Jul 24 14:18:21.749281 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:21.749202 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9630a2f5-785b-469a-9a24-af6e963d48e9" path="/var/lib/kubelet/pods/9630a2f5-785b-469a-9a24-af6e963d48e9/volumes" Jul 24 14:18:24.841083 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:24.841050 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:26.006659 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.006624 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm"] Jul 24 14:18:26.007073 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.006872 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" containerID="cri-o://414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63" gracePeriod=30 Jul 24 14:18:26.029025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.028969 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:26.262570 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.262477 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:26.529893 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.529804 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:26.848802 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.848769 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj"] Jul 24 14:18:26.849164 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849151 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9630a2f5-785b-469a-9a24-af6e963d48e9" containerName="payload-pre-processing" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849166 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9630a2f5-785b-469a-9a24-af6e963d48e9" containerName="payload-pre-processing" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849188 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849195 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849202 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849208 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849216 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5461be99-43be-4dc0-b1f7-33cf2c6001b7" containerName="payload-processing" Jul 24 14:18:26.849218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849221 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5461be99-43be-4dc0-b1f7-33cf2c6001b7" containerName="payload-processing" Jul 24 14:18:26.849432 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849283 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" Jul 24 14:18:26.849432 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849290 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9630a2f5-785b-469a-9a24-af6e963d48e9" containerName="payload-pre-processing" Jul 24 14:18:26.849432 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.849298 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5461be99-43be-4dc0-b1f7-33cf2c6001b7" containerName="payload-processing" Jul 24 14:18:26.853644 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.853610 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854292 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854264 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-certs\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854421 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854299 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-data\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854421 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854379 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-socket\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854544 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854434 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-envoy\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854544 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854463 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-token\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854544 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854494 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-credential-socket\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854544 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854521 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13de7fd1-61b4-4949-9d9d-fc120958fc30-istiod-ca-cert\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854648 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-podinfo\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.854687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.854669 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzdnz\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-kube-api-access-jzdnz\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.856835 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.856817 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8fc7075e-openshift-default-dockercfg-vcnzx\"" Jul 24 14:18:26.865621 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.865597 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj"] Jul 24 14:18:26.873714 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.873679 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:26.955252 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955221 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-certs\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955252 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-data\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955508 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955312 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-socket\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955508 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955472 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-envoy\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955520 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-token\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955558 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-credential-socket\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955582 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13de7fd1-61b4-4949-9d9d-fc120958fc30-istiod-ca-cert\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955671 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-certs\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955676 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-podinfo\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzdnz\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-kube-api-access-jzdnz\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955709 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-data\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.955781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955758 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-socket\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.956037 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.955919 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-credential-socket\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.956281 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.956262 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13de7fd1-61b4-4949-9d9d-fc120958fc30-istiod-ca-cert\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.958000 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.957955 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-envoy\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.958324 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.958308 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-podinfo\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.964545 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.964524 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-token\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.964856 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.964836 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzdnz\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-kube-api-access-jzdnz\") pod \"e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:26.973053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:26.973023 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:27.166584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.166483 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:27.310452 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.310426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj"] Jul 24 14:18:27.312417 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:18:27.312389 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13de7fd1_61b4_4949_9d9d_fc120958fc30.slice/crio-e429e388eadce672cb6070cd64d074d5d74bfcab00ee2b9bd110b260a212c786 WatchSource:0}: Error finding container e429e388eadce672cb6070cd64d074d5d74bfcab00ee2b9bd110b260a212c786: Status 404 returned error can't find the container with id e429e388eadce672cb6070cd64d074d5d74bfcab00ee2b9bd110b260a212c786 Jul 24 14:18:27.314581 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.314548 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:27.314698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.314624 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:27.314698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.314664 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:27.833456 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.833415 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" event={"ID":"13de7fd1-61b4-4949-9d9d-fc120958fc30","Type":"ContainerStarted","Data":"97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e"} Jul 24 14:18:27.833633 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.833474 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" event={"ID":"13de7fd1-61b4-4949-9d9d-fc120958fc30","Type":"ContainerStarted","Data":"e429e388eadce672cb6070cd64d074d5d74bfcab00ee2b9bd110b260a212c786"} Jul 24 14:18:27.870442 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:27.870379 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podStartSLOduration=1.870360469 podStartE2EDuration="1.870360469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:18:27.868269271 +0000 UTC m=+1994.663612002" watchObservedRunningTime="2026-07-24 14:18:27.870360469 +0000 UTC m=+1994.665703202" Jul 24 14:18:28.167396 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:28.167305 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:28.172463 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:28.172436 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:28.838346 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:28.838310 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:28.839397 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:28.839380 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:29.217644 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:29.217539 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:29.317275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:29.317243 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:29.415386 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:29.415345 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:29.556872 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:29.556836 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:30.000584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.000482 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:30.101236 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.101206 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:18:30.109192 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.109166 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6"] Jul 24 14:18:30.109598 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.109586 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e29f99c-f0f1-4e89-8213-8db00b0cdc1f" containerName="istio-proxy" Jul 24 14:18:30.112682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.112662 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.115430 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.115409 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-8fc7075e\"" Jul 24 14:18:30.115567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.115511 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-8fc7075e-dockercfg-28t8j\"" Jul 24 14:18:30.124965 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.124938 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6"] Jul 24 14:18:30.141306 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.141271 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk"] Jul 24 14:18:30.144763 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.144747 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.159139 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.159111 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk"] Jul 24 14:18:30.185372 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.185341 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b857d7aa-0f19-43d8-9115-4b624c2d96d5-plugins-config-volume\") pod \"payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.185372 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.185377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh96q\" (UniqueName: \"kubernetes.io/projected/04684c1b-5229-4a43-842f-025de9704ca9-kube-api-access-sh96q\") pod \"payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.185588 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.185406 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqtph\" (UniqueName: \"kubernetes.io/projected/b857d7aa-0f19-43d8-9115-4b624c2d96d5-kube-api-access-fqtph\") pod \"payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.185588 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.185465 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04684c1b-5229-4a43-842f-025de9704ca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.286702 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.286616 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b857d7aa-0f19-43d8-9115-4b624c2d96d5-plugins-config-volume\") pod \"payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.286702 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.286660 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh96q\" (UniqueName: \"kubernetes.io/projected/04684c1b-5229-4a43-842f-025de9704ca9-kube-api-access-sh96q\") pod \"payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.286702 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.286698 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqtph\" (UniqueName: \"kubernetes.io/projected/b857d7aa-0f19-43d8-9115-4b624c2d96d5-kube-api-access-fqtph\") pod \"payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.287307 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.286741 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04684c1b-5229-4a43-842f-025de9704ca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.287416 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.287396 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b857d7aa-0f19-43d8-9115-4b624c2d96d5-plugins-config-volume\") pod \"payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.287473 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.287457 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04684c1b-5229-4a43-842f-025de9704ca9-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.298535 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.298507 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh96q\" (UniqueName: \"kubernetes.io/projected/04684c1b-5229-4a43-842f-025de9704ca9-kube-api-access-sh96q\") pod \"payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.298664 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.298578 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqtph\" (UniqueName: \"kubernetes.io/projected/b857d7aa-0f19-43d8-9115-4b624c2d96d5-kube-api-access-fqtph\") pod \"payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.423869 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.423838 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.455042 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.455012 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.592062 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.591972 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6"] Jul 24 14:18:30.593537 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:18:30.593509 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04684c1b_5229_4a43_842f_025de9704ca9.slice/crio-9a25358816390e8d245598ba2ea0eefabf052e02a16fe44293c373cf165061e8 WatchSource:0}: Error finding container 9a25358816390e8d245598ba2ea0eefabf052e02a16fe44293c373cf165061e8: Status 404 returned error can't find the container with id 9a25358816390e8d245598ba2ea0eefabf052e02a16fe44293c373cf165061e8 Jul 24 14:18:30.626307 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.626282 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk"] Jul 24 14:18:30.631193 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:18:30.631167 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb857d7aa_0f19_43d8_9115_4b624c2d96d5.slice/crio-6105b3382044af106384cf2419f5fd60c83a936ce1239e7de53091eb5abf0807 WatchSource:0}: Error finding container 6105b3382044af106384cf2419f5fd60c83a936ce1239e7de53091eb5abf0807: Status 404 returned error can't find the container with id 6105b3382044af106384cf2419f5fd60c83a936ce1239e7de53091eb5abf0807 Jul 24 14:18:30.850382 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.850277 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" event={"ID":"b857d7aa-0f19-43d8-9115-4b624c2d96d5","Type":"ContainerStarted","Data":"7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d"} Jul 24 14:18:30.850382 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.850329 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" event={"ID":"b857d7aa-0f19-43d8-9115-4b624c2d96d5","Type":"ContainerStarted","Data":"6105b3382044af106384cf2419f5fd60c83a936ce1239e7de53091eb5abf0807"} Jul 24 14:18:30.850617 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.850448 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:30.851876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.851853 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" event={"ID":"04684c1b-5229-4a43-842f-025de9704ca9","Type":"ContainerStarted","Data":"f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a"} Jul 24 14:18:30.852031 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.851892 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" event={"ID":"04684c1b-5229-4a43-842f-025de9704ca9","Type":"ContainerStarted","Data":"9a25358816390e8d245598ba2ea0eefabf052e02a16fe44293c373cf165061e8"} Jul 24 14:18:30.852031 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.851919 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:18:30.867237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.867169 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" podStartSLOduration=0.867148533 podStartE2EDuration="867.148533ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:18:30.866123342 +0000 UTC m=+1997.661466071" watchObservedRunningTime="2026-07-24 14:18:30.867148533 +0000 UTC m=+1997.662491264" Jul 24 14:18:30.884239 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:30.884183 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" podStartSLOduration=0.884164505 podStartE2EDuration="884.164505ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:18:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:18:30.882038604 +0000 UTC m=+1997.677381338" watchObservedRunningTime="2026-07-24 14:18:30.884164505 +0000 UTC m=+1997.679507236" Jul 24 14:18:31.258094 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.258069 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:18:31.299498 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299462 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-socket\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299516 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51df5696-7d52-4cfd-a5f7-016663b4dc94-istiod-ca-cert\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299570 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-certs\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299603 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-data\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299633 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-token\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299672 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnvkr\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-kube-api-access-qnvkr\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.299735 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-podinfo\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.300451 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.300408 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:31.300593 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.300574 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-data" (OuterVolumeSpecName: "istio-data") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:31.300658 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.300597 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:31.300957 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.300934 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51df5696-7d52-4cfd-a5f7-016663b4dc94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:18:31.301210 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.301034 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-envoy\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.301210 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.301078 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-credential-socket\") pod \"51df5696-7d52-4cfd-a5f7-016663b4dc94\" (UID: \"51df5696-7d52-4cfd-a5f7-016663b4dc94\") " Jul 24 14:18:31.302029 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.301953 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.302329 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.302305 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:31.302442 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.302002 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51df5696-7d52-4cfd-a5f7-016663b4dc94-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.302442 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.302361 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.302442 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.302375 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.304472 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.304412 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:18:31.304583 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.304568 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:18:31.304962 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.304942 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-kube-api-access-qnvkr" (OuterVolumeSpecName: "kube-api-access-qnvkr") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "kube-api-access-qnvkr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:18:31.305228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.305211 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-token" (OuterVolumeSpecName: "istio-token") pod "51df5696-7d52-4cfd-a5f7-016663b4dc94" (UID: "51df5696-7d52-4cfd-a5f7-016663b4dc94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:18:31.403089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.402968 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.403089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.403032 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnvkr\" (UniqueName: \"kubernetes.io/projected/51df5696-7d52-4cfd-a5f7-016663b4dc94-kube-api-access-qnvkr\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.403089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.403047 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.403089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.403060 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.403089 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.403074 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51df5696-7d52-4cfd-a5f7-016663b4dc94-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:18:31.858596 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.858558 2555 generic.go:356] "Generic (PLEG): container finished" podID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerID="414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63" exitCode=0 Jul 24 14:18:31.858784 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.858650 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" Jul 24 14:18:31.858784 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.858665 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" event={"ID":"51df5696-7d52-4cfd-a5f7-016663b4dc94","Type":"ContainerDied","Data":"414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63"} Jul 24 14:18:31.858784 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.858761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm" event={"ID":"51df5696-7d52-4cfd-a5f7-016663b4dc94","Type":"ContainerDied","Data":"54ca168a4c26d58068ebe305888f96d660c59a750b6ab32eda7062ce8e472005"} Jul 24 14:18:31.858963 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.858793 2555 scope.go:117] "RemoveContainer" containerID="414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63" Jul 24 14:18:31.867924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.867905 2555 scope.go:117] "RemoveContainer" containerID="6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a" Jul 24 14:18:31.876679 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.876661 2555 scope.go:117] "RemoveContainer" containerID="414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63" Jul 24 14:18:31.877140 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:18:31.877119 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63\": container with ID starting with 414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63 not found: ID does not exist" containerID="414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63" Jul 24 14:18:31.877234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.877160 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63"} err="failed to get container status \"414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63\": rpc error: code = NotFound desc = could not find container \"414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63\": container with ID starting with 414550a1827594a421c89158b0406b231d5c41d878d9150a015b460c49d06c63 not found: ID does not exist" Jul 24 14:18:31.877234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.877180 2555 scope.go:117] "RemoveContainer" containerID="6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a" Jul 24 14:18:31.877481 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:18:31.877462 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a\": container with ID starting with 6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a not found: ID does not exist" containerID="6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a" Jul 24 14:18:31.877530 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.877490 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a"} err="failed to get container status \"6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a\": rpc error: code = NotFound desc = could not find container \"6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a\": container with ID starting with 6c1b9fcf6be448b50a15216b921509b90b760bcb20b7e512503c4855812ecb7a not found: ID does not exist" Jul 24 14:18:31.897229 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.897195 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm"] Jul 24 14:18:31.909059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:31.909027 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1f2fc6f2-openshift-default-6c979fc776-jtmqm"] Jul 24 14:18:33.748994 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:33.748949 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" path="/var/lib/kubelet/pods/51df5696-7d52-4cfd-a5f7-016663b4dc94/volumes" Jul 24 14:18:41.860870 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:41.860823 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" podUID="04684c1b-5229-4a43-842f-025de9704ca9" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.58:9004: connect: connection refused" Jul 24 14:18:41.861327 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:41.861106 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:18:43.839801 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.839756 2555 patch_prober.go:28] interesting pod/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 14:18:43.840231 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.839825 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 14:18:43.932475 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.932439 2555 generic.go:356] "Generic (PLEG): container finished" podID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerID="97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e" exitCode=0 Jul 24 14:18:43.932645 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.932512 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" event={"ID":"13de7fd1-61b4-4949-9d9d-fc120958fc30","Type":"ContainerDied","Data":"97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e"} Jul 24 14:18:43.932909 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.932896 2555 scope.go:117] "RemoveContainer" containerID="97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e" Jul 24 14:18:43.933970 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.933936 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:43.934049 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.934018 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:43.934088 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:43.934061 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:18:44.939324 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:44.939289 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" event={"ID":"13de7fd1-61b4-4949-9d9d-fc120958fc30","Type":"ContainerStarted","Data":"bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8"} Jul 24 14:18:45.167538 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:45.167499 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:45.167737 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:45.167716 2555 patch_prober.go:28] interesting pod/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 14:18:45.167784 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:45.167762 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 14:18:46.167405 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:46.167365 2555 patch_prober.go:28] interesting pod/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 14:18:46.167812 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:46.167425 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 14:18:47.167602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:47.167563 2555 patch_prober.go:28] interesting pod/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 14:18:47.168100 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:47.167625 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 14:18:48.167213 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:48.167170 2555 patch_prober.go:28] interesting pod/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 14:18:48.167402 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:48.167239 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 14:18:49.171304 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:49.171269 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:49.171680 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:49.171503 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:49.172329 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:49.172311 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:18:51.859716 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:18:51.859685 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:19:04.560051 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.560016 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:04.599318 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.599281 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk"] Jul 24 14:19:04.599605 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.599575 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" podUID="b857d7aa-0f19-43d8-9115-4b624c2d96d5" containerName="payload-processing" containerID="cri-o://7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d" gracePeriod=30 Jul 24 14:19:04.604965 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.604936 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6"] Jul 24 14:19:04.605266 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.605218 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" podUID="04684c1b-5229-4a43-842f-025de9704ca9" containerName="payload-pre-processing" containerID="cri-o://f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a" gracePeriod=30 Jul 24 14:19:04.637657 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.637628 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:04.737800 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.737720 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:04.902809 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.902782 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:19:04.903917 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.903894 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:19:04.938499 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.938469 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b857d7aa-0f19-43d8-9115-4b624c2d96d5-plugins-config-volume\") pod \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " Jul 24 14:19:04.938667 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.938545 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04684c1b-5229-4a43-842f-025de9704ca9-plugins-config-volume\") pod \"04684c1b-5229-4a43-842f-025de9704ca9\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " Jul 24 14:19:04.938667 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.938602 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqtph\" (UniqueName: \"kubernetes.io/projected/b857d7aa-0f19-43d8-9115-4b624c2d96d5-kube-api-access-fqtph\") pod \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\" (UID: \"b857d7aa-0f19-43d8-9115-4b624c2d96d5\") " Jul 24 14:19:04.938667 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.938618 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh96q\" (UniqueName: \"kubernetes.io/projected/04684c1b-5229-4a43-842f-025de9704ca9-kube-api-access-sh96q\") pod \"04684c1b-5229-4a43-842f-025de9704ca9\" (UID: \"04684c1b-5229-4a43-842f-025de9704ca9\") " Jul 24 14:19:04.939022 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.938940 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b857d7aa-0f19-43d8-9115-4b624c2d96d5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b857d7aa-0f19-43d8-9115-4b624c2d96d5" (UID: "b857d7aa-0f19-43d8-9115-4b624c2d96d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:19:04.939147 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.939083 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b857d7aa-0f19-43d8-9115-4b624c2d96d5-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:04.939283 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.939164 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04684c1b-5229-4a43-842f-025de9704ca9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "04684c1b-5229-4a43-842f-025de9704ca9" (UID: "04684c1b-5229-4a43-842f-025de9704ca9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:19:04.941511 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.941483 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b857d7aa-0f19-43d8-9115-4b624c2d96d5-kube-api-access-fqtph" (OuterVolumeSpecName: "kube-api-access-fqtph") pod "b857d7aa-0f19-43d8-9115-4b624c2d96d5" (UID: "b857d7aa-0f19-43d8-9115-4b624c2d96d5"). InnerVolumeSpecName "kube-api-access-fqtph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:19:04.941973 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:04.941946 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04684c1b-5229-4a43-842f-025de9704ca9-kube-api-access-sh96q" (OuterVolumeSpecName: "kube-api-access-sh96q") pod "04684c1b-5229-4a43-842f-025de9704ca9" (UID: "04684c1b-5229-4a43-842f-025de9704ca9"). InnerVolumeSpecName "kube-api-access-sh96q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:19:05.040010 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.039940 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04684c1b-5229-4a43-842f-025de9704ca9-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:05.040010 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.039975 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqtph\" (UniqueName: \"kubernetes.io/projected/b857d7aa-0f19-43d8-9115-4b624c2d96d5-kube-api-access-fqtph\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:05.040010 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.040015 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh96q\" (UniqueName: \"kubernetes.io/projected/04684c1b-5229-4a43-842f-025de9704ca9-kube-api-access-sh96q\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:05.049043 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.049000 2555 generic.go:356] "Generic (PLEG): container finished" podID="b857d7aa-0f19-43d8-9115-4b624c2d96d5" containerID="7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d" exitCode=0 Jul 24 14:19:05.049232 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.049048 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" Jul 24 14:19:05.049232 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.049083 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" event={"ID":"b857d7aa-0f19-43d8-9115-4b624c2d96d5","Type":"ContainerDied","Data":"7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d"} Jul 24 14:19:05.049232 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.049137 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk" event={"ID":"b857d7aa-0f19-43d8-9115-4b624c2d96d5","Type":"ContainerDied","Data":"6105b3382044af106384cf2419f5fd60c83a936ce1239e7de53091eb5abf0807"} Jul 24 14:19:05.049232 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.049166 2555 scope.go:117] "RemoveContainer" containerID="7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d" Jul 24 14:19:05.050418 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.050389 2555 generic.go:356] "Generic (PLEG): container finished" podID="04684c1b-5229-4a43-842f-025de9704ca9" containerID="f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a" exitCode=0 Jul 24 14:19:05.050498 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.050440 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" Jul 24 14:19:05.050498 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.050479 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" event={"ID":"04684c1b-5229-4a43-842f-025de9704ca9","Type":"ContainerDied","Data":"f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a"} Jul 24 14:19:05.050597 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.050529 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6" event={"ID":"04684c1b-5229-4a43-842f-025de9704ca9","Type":"ContainerDied","Data":"9a25358816390e8d245598ba2ea0eefabf052e02a16fe44293c373cf165061e8"} Jul 24 14:19:05.060380 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.060356 2555 scope.go:117] "RemoveContainer" containerID="7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d" Jul 24 14:19:05.060879 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:19:05.060854 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d\": container with ID starting with 7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d not found: ID does not exist" containerID="7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d" Jul 24 14:19:05.061021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.060889 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d"} err="failed to get container status \"7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d\": rpc error: code = NotFound desc = could not find container \"7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d\": container with ID starting with 7c24b743875aebc4ce82ab0792859e24695c2bf445f4d8a222cd88fc3053524d not found: ID does not exist" Jul 24 14:19:05.061021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.060924 2555 scope.go:117] "RemoveContainer" containerID="f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a" Jul 24 14:19:05.070470 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.070452 2555 scope.go:117] "RemoveContainer" containerID="f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a" Jul 24 14:19:05.070731 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:19:05.070710 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a\": container with ID starting with f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a not found: ID does not exist" containerID="f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a" Jul 24 14:19:05.070780 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.070739 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a"} err="failed to get container status \"f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a\": rpc error: code = NotFound desc = could not find container \"f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a\": container with ID starting with f9c3439df34d3915763e3d7787748f36c8165134a90eea0c3b524c1041f0d32a not found: ID does not exist" Jul 24 14:19:05.080811 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.080783 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk"] Jul 24 14:19:05.087955 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.087925 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-8fc7075e-679f79b654-sw6gk"] Jul 24 14:19:05.101002 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.100951 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6"] Jul 24 14:19:05.104626 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.104598 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-8fc7075e-764c57996-mcpm6"] Jul 24 14:19:05.749169 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.749134 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04684c1b-5229-4a43-842f-025de9704ca9" path="/var/lib/kubelet/pods/04684c1b-5229-4a43-842f-025de9704ca9/volumes" Jul 24 14:19:05.749552 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:05.749485 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b857d7aa-0f19-43d8-9115-4b624c2d96d5" path="/var/lib/kubelet/pods/b857d7aa-0f19-43d8-9115-4b624c2d96d5/volumes" Jul 24 14:19:09.185379 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:09.185346 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:09.341045 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:09.340966 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj"] Jul 24 14:19:09.341249 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:09.341225 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" containerID="cri-o://bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8" gracePeriod=30 Jul 24 14:19:09.366517 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:09.366481 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:09.605740 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:09.605702 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:10.116374 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:10.116338 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:10.501259 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:10.501168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:10.600149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:10.600119 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:11.796355 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:11.796275 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:14.609744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.609681 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:19:14.628789 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.628760 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzdnz\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-kube-api-access-jzdnz\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.628789 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.628792 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-token\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.628835 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-certs\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.628860 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-podinfo\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.628920 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-credential-socket\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.628952 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-envoy\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629039 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13de7fd1-61b4-4949-9d9d-fc120958fc30-istiod-ca-cert\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629106 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-data\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629137 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:19:14.629173 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629168 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-socket\") pod \"13de7fd1-61b4-4949-9d9d-fc120958fc30\" (UID: \"13de7fd1-61b4-4949-9d9d-fc120958fc30\") " Jul 24 14:19:14.629737 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629516 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13de7fd1-61b4-4949-9d9d-fc120958fc30-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:19:14.629737 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629611 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13de7fd1-61b4-4949-9d9d-fc120958fc30-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.629737 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629629 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.629903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629747 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:19:14.629903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.629890 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:19:14.631115 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.630205 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-data" (OuterVolumeSpecName: "istio-data") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:19:14.631510 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.631485 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-token" (OuterVolumeSpecName: "istio-token") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:19:14.631770 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.631746 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:19:14.631855 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.631813 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-kube-api-access-jzdnz" (OuterVolumeSpecName: "kube-api-access-jzdnz") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "kube-api-access-jzdnz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:19:14.632006 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.631963 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13de7fd1-61b4-4949-9d9d-fc120958fc30" (UID: "13de7fd1-61b4-4949-9d9d-fc120958fc30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:19:14.730572 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730535 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzdnz\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-kube-api-access-jzdnz\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.730572 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730567 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.730572 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730576 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.730572 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730584 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.730906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730593 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.730906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730603 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:14.730906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:14.730612 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13de7fd1-61b4-4949-9d9d-fc120958fc30-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:19:15.105600 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.105566 2555 generic.go:356] "Generic (PLEG): container finished" podID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerID="bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8" exitCode=0 Jul 24 14:19:15.105782 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.105617 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" Jul 24 14:19:15.105782 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.105635 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" event={"ID":"13de7fd1-61b4-4949-9d9d-fc120958fc30","Type":"ContainerDied","Data":"bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8"} Jul 24 14:19:15.105782 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.105676 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj" event={"ID":"13de7fd1-61b4-4949-9d9d-fc120958fc30","Type":"ContainerDied","Data":"e429e388eadce672cb6070cd64d074d5d74bfcab00ee2b9bd110b260a212c786"} Jul 24 14:19:15.105782 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.105697 2555 scope.go:117] "RemoveContainer" containerID="bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8" Jul 24 14:19:15.115764 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.115733 2555 scope.go:117] "RemoveContainer" containerID="97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e" Jul 24 14:19:15.124090 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.124066 2555 scope.go:117] "RemoveContainer" containerID="bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8" Jul 24 14:19:15.124368 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:19:15.124349 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8\": container with ID starting with bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8 not found: ID does not exist" containerID="bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8" Jul 24 14:19:15.124455 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.124382 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8"} err="failed to get container status \"bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8\": rpc error: code = NotFound desc = could not find container \"bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8\": container with ID starting with bc7a7f9dd539c68c1d2c6e45d2a2b505ba555b276150053099de5897019233d8 not found: ID does not exist" Jul 24 14:19:15.124455 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.124409 2555 scope.go:117] "RemoveContainer" containerID="97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e" Jul 24 14:19:15.124684 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:19:15.124667 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e\": container with ID starting with 97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e not found: ID does not exist" containerID="97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e" Jul 24 14:19:15.124729 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.124691 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e"} err="failed to get container status \"97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e\": rpc error: code = NotFound desc = could not find container \"97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e\": container with ID starting with 97c6b88b6299c16bdb14a8979ead5e396ed9a0e346808bc52653623a290e621e not found: ID does not exist" Jul 24 14:19:15.128843 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.128817 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj"] Jul 24 14:19:15.132685 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.132661 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8fc7075e-openshift-default-5f7cc97864-7htgj"] Jul 24 14:19:15.750305 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:15.750272 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" path="/var/lib/kubelet/pods/13de7fd1-61b4-4949-9d9d-fc120958fc30/volumes" Jul 24 14:19:17.726561 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:17.726530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:17.828113 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:17.828076 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:17.928236 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:17.928201 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:18.029134 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:18.029013 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:18.443582 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:18.443553 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:18.540091 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:18.540042 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:44.413938 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:44.413856 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:50.548648 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:50.548605 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:50.655556 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:50.655524 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:19:50.749538 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:19:50.749505 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:06.694319 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:06.694280 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:12.733744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:12.733711 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:12.949320 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:12.949285 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:13.066913 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:13.066879 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:23.480573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:23.480538 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:24.383779 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:24.383739 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:30.942174 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:30.942131 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:31.047326 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:31.047281 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:31.144197 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:31.144158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:31.241157 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:31.241068 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:32.097584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:32.097549 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:32.180366 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:32.180335 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:33.871278 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:33.871232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:39.901027 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:39.900992 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:40.002874 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:40.002837 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:40.603275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:40.603242 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:46.853395 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:46.853344 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:46.957818 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:46.957772 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:47.117072 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.116962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:47.605913 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.605870 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d"] Jul 24 14:20:47.606439 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606423 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" Jul 24 14:20:47.606492 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606442 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" Jul 24 14:20:47.606492 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606470 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" Jul 24 14:20:47.606492 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606479 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" Jul 24 14:20:47.606584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606491 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04684c1b-5229-4a43-842f-025de9704ca9" containerName="payload-pre-processing" Jul 24 14:20:47.606584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606499 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="04684c1b-5229-4a43-842f-025de9704ca9" containerName="payload-pre-processing" Jul 24 14:20:47.606584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606513 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b857d7aa-0f19-43d8-9115-4b624c2d96d5" containerName="payload-processing" Jul 24 14:20:47.606584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606522 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b857d7aa-0f19-43d8-9115-4b624c2d96d5" containerName="payload-processing" Jul 24 14:20:47.606584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606536 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" Jul 24 14:20:47.606584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606544 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" Jul 24 14:20:47.606765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606628 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" Jul 24 14:20:47.606765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606641 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="13de7fd1-61b4-4949-9d9d-fc120958fc30" containerName="istio-proxy" Jul 24 14:20:47.606765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606653 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" Jul 24 14:20:47.606765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606662 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" Jul 24 14:20:47.606765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606673 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="04684c1b-5229-4a43-842f-025de9704ca9" containerName="payload-pre-processing" Jul 24 14:20:47.606765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.606682 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b857d7aa-0f19-43d8-9115-4b624c2d96d5" containerName="payload-processing" Jul 24 14:20:47.610090 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.610068 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.613351 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.613330 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-89fed8b8-openshift-default-dockercfg-6tddq\"" Jul 24 14:20:47.620388 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.620363 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:47.700330 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700290 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-envoy\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700373 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-podinfo\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700473 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-certs\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700524 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-data\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700565 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2482056d-81c1-4876-8e3c-5fe82a92743f-istiod-ca-cert\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700627 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-socket\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-token\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-credential-socket\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.700919 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.700728 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r7j4\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-kube-api-access-8r7j4\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.730317 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.730266 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d"] Jul 24 14:20:47.801641 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801605 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2482056d-81c1-4876-8e3c-5fe82a92743f-istiod-ca-cert\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.801857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801672 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-socket\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.801857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801707 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-token\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.801857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-credential-socket\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.801857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801763 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r7j4\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-kube-api-access-8r7j4\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.801857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801801 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-envoy\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.802169 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801858 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-podinfo\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.802169 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-certs\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.802169 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.801924 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-data\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.802631 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.802546 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2482056d-81c1-4876-8e3c-5fe82a92743f-istiod-ca-cert\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.802631 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.802551 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-credential-socket\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.803103 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.803069 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-certs\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.803103 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.803087 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-data\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.803303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.803284 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-socket\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.805120 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.805091 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-envoy\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.805588 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.805569 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-podinfo\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.815916 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.815869 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-token\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.830759 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.830728 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r7j4\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-kube-api-access-8r7j4\") pod \"e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:47.922891 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:47.922770 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:48.097072 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.097040 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d"] Jul 24 14:20:48.098293 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:20:48.098267 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2482056d_81c1_4876_8e3c_5fe82a92743f.slice/crio-ba68edf49b6d4ea40f88f030c80972f383ec4828bd19cfd8b5bf48704935d613 WatchSource:0}: Error finding container ba68edf49b6d4ea40f88f030c80972f383ec4828bd19cfd8b5bf48704935d613: Status 404 returned error can't find the container with id ba68edf49b6d4ea40f88f030c80972f383ec4828bd19cfd8b5bf48704935d613 Jul 24 14:20:48.100640 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.100596 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:20:48.100744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.100664 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:20:48.100744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.100728 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:20:48.101018 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.100962 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:48.573649 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.573608 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" event={"ID":"2482056d-81c1-4876-8e3c-5fe82a92743f","Type":"ContainerStarted","Data":"6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1"} Jul 24 14:20:48.573818 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.573680 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" event={"ID":"2482056d-81c1-4876-8e3c-5fe82a92743f","Type":"ContainerStarted","Data":"ba68edf49b6d4ea40f88f030c80972f383ec4828bd19cfd8b5bf48704935d613"} Jul 24 14:20:48.596874 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.596811 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podStartSLOduration=1.596790369 podStartE2EDuration="1.596790369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:20:48.596266529 +0000 UTC m=+2135.391609263" watchObservedRunningTime="2026-07-24 14:20:48.596790369 +0000 UTC m=+2135.392133101" Jul 24 14:20:48.923755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:48.923651 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:49.929291 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:49.929251 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:50.586632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:50.586577 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:50.587604 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:50.587572 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:20:50.638421 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:50.638390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:50.742112 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:50.742075 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:50.839832 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:50.839746 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:50.986496 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:50.986466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:51.326802 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.326764 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:51.426026 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.425973 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:51.445268 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.445232 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf"] Jul 24 14:20:51.445631 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.445619 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" Jul 24 14:20:51.445678 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.445633 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="51df5696-7d52-4cfd-a5f7-016663b4dc94" containerName="istio-proxy" Jul 24 14:20:51.448961 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.448938 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.451651 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.451622 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh"] Jul 24 14:20:51.453322 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.453301 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-89fed8b8\"" Jul 24 14:20:51.453484 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.453302 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-89fed8b8-dockercfg-bmnb6\"" Jul 24 14:20:51.455147 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.455130 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.462078 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.462052 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf"] Jul 24 14:20:51.471824 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.471793 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh"] Jul 24 14:20:51.539060 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.539010 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czlp2\" (UniqueName: \"kubernetes.io/projected/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-kube-api-access-czlp2\") pod \"payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.539267 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.539081 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-plugins-config-volume\") pod \"payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.539267 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.539140 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8cx7\" (UniqueName: \"kubernetes.io/projected/1bbeda67-1f21-4c07-8514-b95527d9ffbb-kube-api-access-w8cx7\") pod \"payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.539382 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.539260 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbeda67-1f21-4c07-8514-b95527d9ffbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.640740 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.640636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czlp2\" (UniqueName: \"kubernetes.io/projected/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-kube-api-access-czlp2\") pod \"payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.640740 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.640679 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-plugins-config-volume\") pod \"payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.640740 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.640728 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8cx7\" (UniqueName: \"kubernetes.io/projected/1bbeda67-1f21-4c07-8514-b95527d9ffbb-kube-api-access-w8cx7\") pod \"payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.641092 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.640780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbeda67-1f21-4c07-8514-b95527d9ffbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.641514 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.641493 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-plugins-config-volume\") pod \"payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.641555 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.641519 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbeda67-1f21-4c07-8514-b95527d9ffbb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.649941 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.649912 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8cx7\" (UniqueName: \"kubernetes.io/projected/1bbeda67-1f21-4c07-8514-b95527d9ffbb-kube-api-access-w8cx7\") pod \"payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.650118 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.650070 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czlp2\" (UniqueName: \"kubernetes.io/projected/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-kube-api-access-czlp2\") pod \"payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.761891 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.761853 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:51.768933 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.768904 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:51.937542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.937507 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf"] Jul 24 14:20:51.939494 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:20:51.939453 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbeda67_1f21_4c07_8514_b95527d9ffbb.slice/crio-552ff2deb3b26c946ef2b32a46574830a551b4aa79188f5723657f149f02c15c WatchSource:0}: Error finding container 552ff2deb3b26c946ef2b32a46574830a551b4aa79188f5723657f149f02c15c: Status 404 returned error can't find the container with id 552ff2deb3b26c946ef2b32a46574830a551b4aa79188f5723657f149f02c15c Jul 24 14:20:51.972290 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:51.972130 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh"] Jul 24 14:20:51.982424 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:20:51.982391 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26949d56_d0f4_47eb_8faa_4e94e73e8ad3.slice/crio-9aa9f078fae4e85d2d375bbe923b7bb20b3a32a824b59648f58f9b2ecb64ada2 WatchSource:0}: Error finding container 9aa9f078fae4e85d2d375bbe923b7bb20b3a32a824b59648f58f9b2ecb64ada2: Status 404 returned error can't find the container with id 9aa9f078fae4e85d2d375bbe923b7bb20b3a32a824b59648f58f9b2ecb64ada2 Jul 24 14:20:52.598877 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.598828 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" event={"ID":"1bbeda67-1f21-4c07-8514-b95527d9ffbb","Type":"ContainerStarted","Data":"2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512"} Jul 24 14:20:52.599431 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.598896 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" event={"ID":"1bbeda67-1f21-4c07-8514-b95527d9ffbb","Type":"ContainerStarted","Data":"552ff2deb3b26c946ef2b32a46574830a551b4aa79188f5723657f149f02c15c"} Jul 24 14:20:52.599431 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.598923 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:20:52.600537 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.600507 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" event={"ID":"26949d56-d0f4-47eb-8faa-4e94e73e8ad3","Type":"ContainerStarted","Data":"0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0"} Jul 24 14:20:52.600674 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.600561 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" event={"ID":"26949d56-d0f4-47eb-8faa-4e94e73e8ad3","Type":"ContainerStarted","Data":"9aa9f078fae4e85d2d375bbe923b7bb20b3a32a824b59648f58f9b2ecb64ada2"} Jul 24 14:20:52.600752 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.600678 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:20:52.616497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.616442 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" podStartSLOduration=1.6164256940000001 podStartE2EDuration="1.616425694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:20:52.614838576 +0000 UTC m=+2139.410181307" watchObservedRunningTime="2026-07-24 14:20:52.616425694 +0000 UTC m=+2139.411768424" Jul 24 14:20:52.632710 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:52.632648 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" podStartSLOduration=1.632630732 podStartE2EDuration="1.632630732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:20:52.63192694 +0000 UTC m=+2139.427269671" watchObservedRunningTime="2026-07-24 14:20:52.632630732 +0000 UTC m=+2139.427973462" Jul 24 14:20:55.125461 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:55.125427 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:55.787608 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:55.787566 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:55.885754 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:55.885719 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:20:56.225387 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:20:56.225356 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:01.965557 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:01.965520 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:02.119602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:02.119571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:03.608466 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:03.608428 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:21:03.609115 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:03.609095 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:21:06.628229 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:06.628181 2555 patch_prober.go:28] interesting pod/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:21:06.628645 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:06.628280 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 14:21:08.689584 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:08.689547 2555 generic.go:356] "Generic (PLEG): container finished" podID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerID="6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1" exitCode=0 Jul 24 14:21:08.690153 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:08.689619 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" event={"ID":"2482056d-81c1-4876-8e3c-5fe82a92743f","Type":"ContainerDied","Data":"6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1"} Jul 24 14:21:08.690153 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:08.690122 2555 scope.go:117] "RemoveContainer" containerID="6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1" Jul 24 14:21:08.691183 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:08.691153 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:08.691267 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:08.691224 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:08.691267 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:08.691253 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:09.697967 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:09.697930 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" event={"ID":"2482056d-81c1-4876-8e3c-5fe82a92743f","Type":"ContainerStarted","Data":"c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112"} Jul 24 14:21:09.923545 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:09.923500 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:21:09.923813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:09.923788 2555 patch_prober.go:28] interesting pod/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:09.923910 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:09.923850 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:10.923779 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:10.923737 2555 patch_prober.go:28] interesting pod/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:10.924244 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:10.923798 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:11.923688 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:11.923651 2555 patch_prober.go:28] interesting pod/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:11.923890 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:11.923710 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:12.834283 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.834243 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:12.867711 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.867661 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh"] Jul 24 14:21:12.867925 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.867908 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" podUID="26949d56-d0f4-47eb-8faa-4e94e73e8ad3" containerName="payload-processing" containerID="cri-o://0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0" gracePeriod=30 Jul 24 14:21:12.876873 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.876838 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf"] Jul 24 14:21:12.877300 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.877266 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" podUID="1bbeda67-1f21-4c07-8514-b95527d9ffbb" containerName="payload-pre-processing" containerID="cri-o://2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512" gracePeriod=30 Jul 24 14:21:12.924887 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.924260 2555 patch_prober.go:28] interesting pod/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:12.924887 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:12.924343 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:13.255636 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.255611 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:21:13.258930 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.258904 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:21:13.449206 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.449104 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8cx7\" (UniqueName: \"kubernetes.io/projected/1bbeda67-1f21-4c07-8514-b95527d9ffbb-kube-api-access-w8cx7\") pod \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " Jul 24 14:21:13.449206 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.449151 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czlp2\" (UniqueName: \"kubernetes.io/projected/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-kube-api-access-czlp2\") pod \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " Jul 24 14:21:13.449206 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.449174 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbeda67-1f21-4c07-8514-b95527d9ffbb-plugins-config-volume\") pod \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\" (UID: \"1bbeda67-1f21-4c07-8514-b95527d9ffbb\") " Jul 24 14:21:13.449516 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.449258 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-plugins-config-volume\") pod \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\" (UID: \"26949d56-d0f4-47eb-8faa-4e94e73e8ad3\") " Jul 24 14:21:13.449677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.449657 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26949d56-d0f4-47eb-8faa-4e94e73e8ad3" (UID: "26949d56-d0f4-47eb-8faa-4e94e73e8ad3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:21:13.449758 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.449670 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bbeda67-1f21-4c07-8514-b95527d9ffbb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1bbeda67-1f21-4c07-8514-b95527d9ffbb" (UID: "1bbeda67-1f21-4c07-8514-b95527d9ffbb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:21:13.451468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.451446 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-kube-api-access-czlp2" (OuterVolumeSpecName: "kube-api-access-czlp2") pod "26949d56-d0f4-47eb-8faa-4e94e73e8ad3" (UID: "26949d56-d0f4-47eb-8faa-4e94e73e8ad3"). InnerVolumeSpecName "kube-api-access-czlp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:21:13.451468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.451456 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bbeda67-1f21-4c07-8514-b95527d9ffbb-kube-api-access-w8cx7" (OuterVolumeSpecName: "kube-api-access-w8cx7") pod "1bbeda67-1f21-4c07-8514-b95527d9ffbb" (UID: "1bbeda67-1f21-4c07-8514-b95527d9ffbb"). InnerVolumeSpecName "kube-api-access-w8cx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:21:13.550010 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.549952 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:13.550010 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.550002 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8cx7\" (UniqueName: \"kubernetes.io/projected/1bbeda67-1f21-4c07-8514-b95527d9ffbb-kube-api-access-w8cx7\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:13.550010 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.550020 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czlp2\" (UniqueName: \"kubernetes.io/projected/26949d56-d0f4-47eb-8faa-4e94e73e8ad3-kube-api-access-czlp2\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:13.550266 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.550032 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbeda67-1f21-4c07-8514-b95527d9ffbb-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:13.732743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.732653 2555 generic.go:356] "Generic (PLEG): container finished" podID="26949d56-d0f4-47eb-8faa-4e94e73e8ad3" containerID="0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0" exitCode=0 Jul 24 14:21:13.732743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.732703 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" Jul 24 14:21:13.732743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.732732 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" event={"ID":"26949d56-d0f4-47eb-8faa-4e94e73e8ad3","Type":"ContainerDied","Data":"0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0"} Jul 24 14:21:13.732969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.732787 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh" event={"ID":"26949d56-d0f4-47eb-8faa-4e94e73e8ad3","Type":"ContainerDied","Data":"9aa9f078fae4e85d2d375bbe923b7bb20b3a32a824b59648f58f9b2ecb64ada2"} Jul 24 14:21:13.732969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.732809 2555 scope.go:117] "RemoveContainer" containerID="0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0" Jul 24 14:21:13.734325 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.734301 2555 generic.go:356] "Generic (PLEG): container finished" podID="1bbeda67-1f21-4c07-8514-b95527d9ffbb" containerID="2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512" exitCode=0 Jul 24 14:21:13.734428 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.734362 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" event={"ID":"1bbeda67-1f21-4c07-8514-b95527d9ffbb","Type":"ContainerDied","Data":"2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512"} Jul 24 14:21:13.734428 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.734408 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" event={"ID":"1bbeda67-1f21-4c07-8514-b95527d9ffbb","Type":"ContainerDied","Data":"552ff2deb3b26c946ef2b32a46574830a551b4aa79188f5723657f149f02c15c"} Jul 24 14:21:13.734526 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.734367 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf" Jul 24 14:21:13.743717 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.743528 2555 scope.go:117] "RemoveContainer" containerID="0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0" Jul 24 14:21:13.744132 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:21:13.743962 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0\": container with ID starting with 0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0 not found: ID does not exist" containerID="0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0" Jul 24 14:21:13.744264 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.744146 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0"} err="failed to get container status \"0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0\": rpc error: code = NotFound desc = could not find container \"0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0\": container with ID starting with 0f6709c52c755509d9747d4bc34ee63daa18118dd480dc6d28c8d6ce7d3523e0 not found: ID does not exist" Jul 24 14:21:13.744264 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.744174 2555 scope.go:117] "RemoveContainer" containerID="2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512" Jul 24 14:21:13.755731 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.755710 2555 scope.go:117] "RemoveContainer" containerID="2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512" Jul 24 14:21:13.756042 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:21:13.756020 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512\": container with ID starting with 2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512 not found: ID does not exist" containerID="2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512" Jul 24 14:21:13.756101 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.756050 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512"} err="failed to get container status \"2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512\": rpc error: code = NotFound desc = could not find container \"2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512\": container with ID starting with 2d50ee1659ac7b6db0ad02029fb63552f2d0d46ab617df10a87f87705c0ce512 not found: ID does not exist" Jul 24 14:21:13.760474 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.760443 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf"] Jul 24 14:21:13.764693 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.764665 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89fed8b8-8f87bdb48-rdwmf"] Jul 24 14:21:13.780314 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.780277 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh"] Jul 24 14:21:13.791625 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.791585 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89fed8b8-7bb9c7dc8b-4twrh"] Jul 24 14:21:13.923641 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.923601 2555 patch_prober.go:28] interesting pod/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 14:21:13.923814 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:13.923687 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 14:21:14.927472 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:14.927441 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:21:14.927897 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:14.927737 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:21:14.928526 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:14.928507 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:21:14.976301 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:14.976261 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:15.749636 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:15.749606 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bbeda67-1f21-4c07-8514-b95527d9ffbb" path="/var/lib/kubelet/pods/1bbeda67-1f21-4c07-8514-b95527d9ffbb/volumes" Jul 24 14:21:15.750007 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:15.749973 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26949d56-d0f4-47eb-8faa-4e94e73e8ad3" path="/var/lib/kubelet/pods/26949d56-d0f4-47eb-8faa-4e94e73e8ad3/volumes" Jul 24 14:21:17.454322 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:17.454291 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:17.880589 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:17.880540 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:23.731397 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:23.731359 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:23.832342 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:23.832303 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:23.985747 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:23.985649 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d"] Jul 24 14:21:23.985897 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:23.985871 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" containerID="cri-o://c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112" gracePeriod=30 Jul 24 14:21:24.013571 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:24.013530 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:24.238201 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:24.238113 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:24.405313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:24.405273 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:24.497304 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:24.497215 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:24.614255 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:24.614217 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:29.246101 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.246071 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:21:29.299560 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299515 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r7j4\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-kube-api-access-8r7j4\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299602 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-data\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299627 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-podinfo\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299646 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-certs\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299667 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-envoy\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299685 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-token\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299717 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-socket\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.299781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299759 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2482056d-81c1-4876-8e3c-5fe82a92743f-istiod-ca-cert\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.300214 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.299803 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-credential-socket\") pod \"2482056d-81c1-4876-8e3c-5fe82a92743f\" (UID: \"2482056d-81c1-4876-8e3c-5fe82a92743f\") " Jul 24 14:21:29.300214 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.300098 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:21:29.300333 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.300305 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:21:29.300419 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.300400 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:21:29.300600 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.300578 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2482056d-81c1-4876-8e3c-5fe82a92743f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:21:29.301255 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.301228 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:21:29.302148 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.302121 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-kube-api-access-8r7j4" (OuterVolumeSpecName: "kube-api-access-8r7j4") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "kube-api-access-8r7j4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:21:29.302443 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.302417 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:21:29.302533 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.302482 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:21:29.302533 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.302506 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2482056d-81c1-4876-8e3c-5fe82a92743f" (UID: "2482056d-81c1-4876-8e3c-5fe82a92743f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:21:29.400889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400847 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r7j4\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-kube-api-access-8r7j4\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.400889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400884 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.400889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400894 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.400889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400901 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.401215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400911 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.401215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400919 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2482056d-81c1-4876-8e3c-5fe82a92743f-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.401215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400926 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.401215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400936 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2482056d-81c1-4876-8e3c-5fe82a92743f-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.401215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.400944 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2482056d-81c1-4876-8e3c-5fe82a92743f-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:21:29.814025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.813974 2555 generic.go:356] "Generic (PLEG): container finished" podID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerID="c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112" exitCode=0 Jul 24 14:21:29.814195 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.814040 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" event={"ID":"2482056d-81c1-4876-8e3c-5fe82a92743f","Type":"ContainerDied","Data":"c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112"} Jul 24 14:21:29.814195 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.814058 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" Jul 24 14:21:29.814195 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.814098 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d" event={"ID":"2482056d-81c1-4876-8e3c-5fe82a92743f","Type":"ContainerDied","Data":"ba68edf49b6d4ea40f88f030c80972f383ec4828bd19cfd8b5bf48704935d613"} Jul 24 14:21:29.814195 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.814122 2555 scope.go:117] "RemoveContainer" containerID="c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112" Jul 24 14:21:29.822936 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.822919 2555 scope.go:117] "RemoveContainer" containerID="6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1" Jul 24 14:21:29.831740 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.831719 2555 scope.go:117] "RemoveContainer" containerID="c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112" Jul 24 14:21:29.832260 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:21:29.832113 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112\": container with ID starting with c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112 not found: ID does not exist" containerID="c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112" Jul 24 14:21:29.832260 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.832155 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112"} err="failed to get container status \"c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112\": rpc error: code = NotFound desc = could not find container \"c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112\": container with ID starting with c3c8618bc4ce655f6f63aea2441db787b2d16e687e13d90d5050fff92ee15112 not found: ID does not exist" Jul 24 14:21:29.832260 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.832179 2555 scope.go:117] "RemoveContainer" containerID="6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1" Jul 24 14:21:29.832615 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:21:29.832552 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1\": container with ID starting with 6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1 not found: ID does not exist" containerID="6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1" Jul 24 14:21:29.832615 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.832583 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1"} err="failed to get container status \"6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1\": rpc error: code = NotFound desc = could not find container \"6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1\": container with ID starting with 6db6581b9af2710cc773aef6c5d4512d32ec86885e3153d83539c2f54f5689d1 not found: ID does not exist" Jul 24 14:21:29.834435 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.834414 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d"] Jul 24 14:21:29.838822 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:29.838801 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-89fed8b8-openshift-default-595c55bf6d-8xp2d"] Jul 24 14:21:30.363016 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:30.362965 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:30.435620 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:30.435584 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:30.521678 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:30.521625 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:30.618586 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:30.618504 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:30.854449 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:30.854419 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:31.749172 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:31.749134 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" path="/var/lib/kubelet/pods/2482056d-81c1-4876-8e3c-5fe82a92743f/volumes" Jul 24 14:21:36.581685 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:36.581654 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:36.654232 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:36.654193 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:43.712218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:43.712182 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:44.028480 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028394 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx"] Jul 24 14:21:44.028765 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028752 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028766 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028781 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26949d56-d0f4-47eb-8faa-4e94e73e8ad3" containerName="payload-processing" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028787 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="26949d56-d0f4-47eb-8faa-4e94e73e8ad3" containerName="payload-processing" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028801 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028807 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028816 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bbeda67-1f21-4c07-8514-b95527d9ffbb" containerName="payload-pre-processing" Jul 24 14:21:44.028821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028821 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bbeda67-1f21-4c07-8514-b95527d9ffbb" containerName="payload-pre-processing" Jul 24 14:21:44.029092 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028879 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="26949d56-d0f4-47eb-8faa-4e94e73e8ad3" containerName="payload-processing" Jul 24 14:21:44.029092 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028887 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" Jul 24 14:21:44.029092 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.028894 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bbeda67-1f21-4c07-8514-b95527d9ffbb" containerName="payload-pre-processing" Jul 24 14:21:44.031884 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.031866 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.035152 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.035132 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-676e8e92-openshift-default-dockercfg-m9t8t\"" Jul 24 14:21:44.047365 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.047333 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx"] Jul 24 14:21:44.055969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.055931 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:44.148898 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.148855 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-credential-socket\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.148898 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.148914 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-certs\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149148 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.148947 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istiod-ca-cert\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149148 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.149036 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-socket\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149148 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.149075 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-data\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149148 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.149092 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-podinfo\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149285 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.149155 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-envoy\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149285 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.149192 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gdvw\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-kube-api-access-4gdvw\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.149285 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.149237 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-token\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.250678 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-token\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.250889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250702 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-credential-socket\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.250889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-certs\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.250889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istiod-ca-cert\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.250889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250818 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-socket\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.250889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250864 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-data\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-podinfo\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250934 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-envoy\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.250968 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gdvw\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-kube-api-access-4gdvw\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.251186 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-credential-socket\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.251352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-certs\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.251372 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-data\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.251459 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-socket\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.251706 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.251681 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istiod-ca-cert\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.253561 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.253536 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-envoy\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.253734 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.253706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-podinfo\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.261547 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.261479 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-token\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.261547 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.261482 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gdvw\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-kube-api-access-4gdvw\") pod \"e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.342869 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.342826 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:44.372873 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.372833 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:44.495239 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.494417 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx"] Jul 24 14:21:44.503150 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.503102 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:44.503268 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.503203 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:44.503268 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.503244 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:44.891132 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.891087 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" event={"ID":"f7b3226d-366c-4e17-8ba1-bd06a2bf8050","Type":"ContainerStarted","Data":"046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84"} Jul 24 14:21:44.891637 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.891157 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" event={"ID":"f7b3226d-366c-4e17-8ba1-bd06a2bf8050","Type":"ContainerStarted","Data":"9e7d99dc734bc2b17c39e80af912719c7c6fe8ad124fb2b94cb0315c0edc34c2"} Jul 24 14:21:44.916441 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:44.916383 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" podStartSLOduration=0.916366604 podStartE2EDuration="916.366604ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:21:44.91535074 +0000 UTC m=+2191.710693472" watchObservedRunningTime="2026-07-24 14:21:44.916366604 +0000 UTC m=+2191.711709334" Jul 24 14:21:45.343272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:45.343233 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:45.348508 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:45.348481 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:45.896706 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:45.896673 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:45.897936 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:45.897911 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:21:47.064570 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.064537 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:47.159889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.159857 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:47.262842 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.262810 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:47.700228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.700173 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:47.798777 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.798748 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:47.809003 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.808955 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z"] Jul 24 14:21:47.809430 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.809417 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2482056d-81c1-4876-8e3c-5fe82a92743f" containerName="istio-proxy" Jul 24 14:21:47.811678 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.811662 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:47.815170 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.815146 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-676e8e92-dockercfg-tl9fb\"" Jul 24 14:21:47.816269 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.816249 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-676e8e92\"" Jul 24 14:21:47.830019 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.829971 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z"] Jul 24 14:21:47.840050 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.840018 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5"] Jul 24 14:21:47.842838 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.842821 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:47.855449 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.855413 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5"] Jul 24 14:21:47.986621 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.986522 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg67z\" (UniqueName: \"kubernetes.io/projected/2d9f8547-386f-4e7a-85db-d17833754a05-kube-api-access-tg67z\") pod \"payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:47.986621 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.986594 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d9f8547-386f-4e7a-85db-d17833754a05-plugins-config-volume\") pod \"payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:47.986872 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.986668 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqm6g\" (UniqueName: \"kubernetes.io/projected/0be5c842-f51a-4696-8012-71fb3a5b3594-kube-api-access-wqm6g\") pod \"payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:47.986872 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:47.986721 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be5c842-f51a-4696-8012-71fb3a5b3594-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.088083 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.088035 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg67z\" (UniqueName: \"kubernetes.io/projected/2d9f8547-386f-4e7a-85db-d17833754a05-kube-api-access-tg67z\") pod \"payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:48.088565 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.088183 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d9f8547-386f-4e7a-85db-d17833754a05-plugins-config-volume\") pod \"payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:48.088565 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.088233 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqm6g\" (UniqueName: \"kubernetes.io/projected/0be5c842-f51a-4696-8012-71fb3a5b3594-kube-api-access-wqm6g\") pod \"payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.088565 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.088281 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be5c842-f51a-4696-8012-71fb3a5b3594-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.089146 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.089122 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d9f8547-386f-4e7a-85db-d17833754a05-plugins-config-volume\") pod \"payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:48.089267 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.089170 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be5c842-f51a-4696-8012-71fb3a5b3594-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.100683 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.100640 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqm6g\" (UniqueName: \"kubernetes.io/projected/0be5c842-f51a-4696-8012-71fb3a5b3594-kube-api-access-wqm6g\") pod \"payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.100827 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.100718 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg67z\" (UniqueName: \"kubernetes.io/projected/2d9f8547-386f-4e7a-85db-d17833754a05-kube-api-access-tg67z\") pod \"payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:48.122533 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.122502 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.154152 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.154120 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:48.280694 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.280642 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z"] Jul 24 14:21:48.283071 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:21:48.283037 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be5c842_f51a_4696_8012_71fb3a5b3594.slice/crio-34f6f0d1056baa9406c073f7a7cdb41ad9416a6275a0dfef9750f29b2e24316c WatchSource:0}: Error finding container 34f6f0d1056baa9406c073f7a7cdb41ad9416a6275a0dfef9750f29b2e24316c: Status 404 returned error can't find the container with id 34f6f0d1056baa9406c073f7a7cdb41ad9416a6275a0dfef9750f29b2e24316c Jul 24 14:21:48.330334 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.330295 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5"] Jul 24 14:21:48.347217 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:21:48.347188 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9f8547_386f_4e7a_85db_d17833754a05.slice/crio-1054467b32e6d5b00a1676387c4e2e77730032ae2df43fe9ffba350ed8819e47 WatchSource:0}: Error finding container 1054467b32e6d5b00a1676387c4e2e77730032ae2df43fe9ffba350ed8819e47: Status 404 returned error can't find the container with id 1054467b32e6d5b00a1676387c4e2e77730032ae2df43fe9ffba350ed8819e47 Jul 24 14:21:48.914709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.914674 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" event={"ID":"2d9f8547-386f-4e7a-85db-d17833754a05","Type":"ContainerStarted","Data":"5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632"} Jul 24 14:21:48.914943 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.914767 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:48.914943 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.914794 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" event={"ID":"2d9f8547-386f-4e7a-85db-d17833754a05","Type":"ContainerStarted","Data":"1054467b32e6d5b00a1676387c4e2e77730032ae2df43fe9ffba350ed8819e47"} Jul 24 14:21:48.916277 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.916252 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" event={"ID":"0be5c842-f51a-4696-8012-71fb3a5b3594","Type":"ContainerStarted","Data":"aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b"} Jul 24 14:21:48.916394 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.916290 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" event={"ID":"0be5c842-f51a-4696-8012-71fb3a5b3594","Type":"ContainerStarted","Data":"34f6f0d1056baa9406c073f7a7cdb41ad9416a6275a0dfef9750f29b2e24316c"} Jul 24 14:21:48.916394 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.916348 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:48.948848 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.948789 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" podStartSLOduration=1.948771928 podStartE2EDuration="1.948771928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:21:48.948301557 +0000 UTC m=+2195.743644290" watchObservedRunningTime="2026-07-24 14:21:48.948771928 +0000 UTC m=+2195.744114678" Jul 24 14:21:48.950174 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:48.950139 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" podStartSLOduration=1.950128128 podStartE2EDuration="1.950128128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:21:48.931365492 +0000 UTC m=+2195.726708221" watchObservedRunningTime="2026-07-24 14:21:48.950128128 +0000 UTC m=+2195.745470874" Jul 24 14:21:51.432857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:51.432817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:21:59.923493 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.923462 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:21:59.923955 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.923538 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:21:59.974815 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.974782 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerID="046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84" exitCode=0 Jul 24 14:21:59.975047 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.974863 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" event={"ID":"f7b3226d-366c-4e17-8ba1-bd06a2bf8050","Type":"ContainerDied","Data":"046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84"} Jul 24 14:21:59.975383 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.975364 2555 scope.go:117] "RemoveContainer" containerID="046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84" Jul 24 14:21:59.976673 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.976629 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:59.976803 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.976732 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:21:59.976803 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:21:59.976783 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:22:00.896775 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:00.896733 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:22:00.982866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:00.982822 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" event={"ID":"f7b3226d-366c-4e17-8ba1-bd06a2bf8050","Type":"ContainerStarted","Data":"96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906"} Jul 24 14:22:01.344109 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:01.344062 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:22:01.345476 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:01.345447 2555 patch_prober.go:28] interesting pod/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 14:22:01.345601 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:01.345503 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 14:22:02.343636 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:02.343588 2555 patch_prober.go:28] interesting pod/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 14:22:02.344102 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:02.343653 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 14:22:03.343355 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:03.343315 2555 patch_prober.go:28] interesting pod/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 14:22:03.343543 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:03.343373 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 14:22:04.343768 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:04.343729 2555 patch_prober.go:28] interesting pod/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 14:22:04.344234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:04.343789 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 14:22:05.347875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:05.347845 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:22:05.348354 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:05.348295 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:22:05.349050 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:05.349032 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:22:16.656514 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:22:16.656460 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:12.830897 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.830861 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:12.856244 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.856209 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5"] Jul 24 14:23:12.856523 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.856474 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" podUID="2d9f8547-386f-4e7a-85db-d17833754a05" containerName="payload-processing" containerID="cri-o://5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632" gracePeriod=30 Jul 24 14:23:12.866166 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.865952 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z"] Jul 24 14:23:12.867375 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.867195 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" podUID="0be5c842-f51a-4696-8012-71fb3a5b3594" containerName="payload-pre-processing" containerID="cri-o://aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b" gracePeriod=30 Jul 24 14:23:12.900327 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.900292 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:12.976402 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:12.976364 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:13.129483 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.129458 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:23:13.159959 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.159919 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:23:13.201660 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.201628 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg67z\" (UniqueName: \"kubernetes.io/projected/2d9f8547-386f-4e7a-85db-d17833754a05-kube-api-access-tg67z\") pod \"2d9f8547-386f-4e7a-85db-d17833754a05\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " Jul 24 14:23:13.201836 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.201696 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d9f8547-386f-4e7a-85db-d17833754a05-plugins-config-volume\") pod \"2d9f8547-386f-4e7a-85db-d17833754a05\" (UID: \"2d9f8547-386f-4e7a-85db-d17833754a05\") " Jul 24 14:23:13.202133 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.202104 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d9f8547-386f-4e7a-85db-d17833754a05-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d9f8547-386f-4e7a-85db-d17833754a05" (UID: "2d9f8547-386f-4e7a-85db-d17833754a05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:23:13.204050 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.204021 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d9f8547-386f-4e7a-85db-d17833754a05-kube-api-access-tg67z" (OuterVolumeSpecName: "kube-api-access-tg67z") pod "2d9f8547-386f-4e7a-85db-d17833754a05" (UID: "2d9f8547-386f-4e7a-85db-d17833754a05"). InnerVolumeSpecName "kube-api-access-tg67z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:23:13.302805 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.302764 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqm6g\" (UniqueName: \"kubernetes.io/projected/0be5c842-f51a-4696-8012-71fb3a5b3594-kube-api-access-wqm6g\") pod \"0be5c842-f51a-4696-8012-71fb3a5b3594\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " Jul 24 14:23:13.303044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.302855 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be5c842-f51a-4696-8012-71fb3a5b3594-plugins-config-volume\") pod \"0be5c842-f51a-4696-8012-71fb3a5b3594\" (UID: \"0be5c842-f51a-4696-8012-71fb3a5b3594\") " Jul 24 14:23:13.303179 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.303162 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg67z\" (UniqueName: \"kubernetes.io/projected/2d9f8547-386f-4e7a-85db-d17833754a05-kube-api-access-tg67z\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:23:13.303236 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.303185 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d9f8547-386f-4e7a-85db-d17833754a05-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:23:13.303286 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.303263 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be5c842-f51a-4696-8012-71fb3a5b3594-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0be5c842-f51a-4696-8012-71fb3a5b3594" (UID: "0be5c842-f51a-4696-8012-71fb3a5b3594"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:23:13.305065 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.305044 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be5c842-f51a-4696-8012-71fb3a5b3594-kube-api-access-wqm6g" (OuterVolumeSpecName: "kube-api-access-wqm6g") pod "0be5c842-f51a-4696-8012-71fb3a5b3594" (UID: "0be5c842-f51a-4696-8012-71fb3a5b3594"). InnerVolumeSpecName "kube-api-access-wqm6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:23:13.387423 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.387322 2555 generic.go:356] "Generic (PLEG): container finished" podID="2d9f8547-386f-4e7a-85db-d17833754a05" containerID="5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632" exitCode=0 Jul 24 14:23:13.387423 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.387381 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" Jul 24 14:23:13.387661 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.387412 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" event={"ID":"2d9f8547-386f-4e7a-85db-d17833754a05","Type":"ContainerDied","Data":"5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632"} Jul 24 14:23:13.387661 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.387511 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5" event={"ID":"2d9f8547-386f-4e7a-85db-d17833754a05","Type":"ContainerDied","Data":"1054467b32e6d5b00a1676387c4e2e77730032ae2df43fe9ffba350ed8819e47"} Jul 24 14:23:13.387661 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.387543 2555 scope.go:117] "RemoveContainer" containerID="5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632" Jul 24 14:23:13.388820 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.388780 2555 generic.go:356] "Generic (PLEG): container finished" podID="0be5c842-f51a-4696-8012-71fb3a5b3594" containerID="aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b" exitCode=0 Jul 24 14:23:13.388885 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.388837 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" Jul 24 14:23:13.388885 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.388867 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" event={"ID":"0be5c842-f51a-4696-8012-71fb3a5b3594","Type":"ContainerDied","Data":"aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b"} Jul 24 14:23:13.389025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.388922 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z" event={"ID":"0be5c842-f51a-4696-8012-71fb3a5b3594","Type":"ContainerDied","Data":"34f6f0d1056baa9406c073f7a7cdb41ad9416a6275a0dfef9750f29b2e24316c"} Jul 24 14:23:13.398620 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.398591 2555 scope.go:117] "RemoveContainer" containerID="5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632" Jul 24 14:23:13.399024 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:23:13.398969 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632\": container with ID starting with 5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632 not found: ID does not exist" containerID="5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632" Jul 24 14:23:13.399131 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.399038 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632"} err="failed to get container status \"5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632\": rpc error: code = NotFound desc = could not find container \"5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632\": container with ID starting with 5672839be62db82925545acab96e83347f0e08662a7c603f1d51b1046d20a632 not found: ID does not exist" Jul 24 14:23:13.399131 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.399062 2555 scope.go:117] "RemoveContainer" containerID="aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b" Jul 24 14:23:13.404591 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.404568 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqm6g\" (UniqueName: \"kubernetes.io/projected/0be5c842-f51a-4696-8012-71fb3a5b3594-kube-api-access-wqm6g\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:23:13.404692 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.404594 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be5c842-f51a-4696-8012-71fb3a5b3594-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:23:13.407507 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.407489 2555 scope.go:117] "RemoveContainer" containerID="aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b" Jul 24 14:23:13.407782 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:23:13.407762 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b\": container with ID starting with aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b not found: ID does not exist" containerID="aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b" Jul 24 14:23:13.407835 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.407791 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b"} err="failed to get container status \"aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b\": rpc error: code = NotFound desc = could not find container \"aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b\": container with ID starting with aa2e3558b50ee2fcae3323dfb07390ffe0da0fd9ac719a52e23767c2251def7b not found: ID does not exist" Jul 24 14:23:13.411797 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.411776 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5"] Jul 24 14:23:13.414819 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.414796 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-676e8e92-5c4c7bf66b-tbwn5"] Jul 24 14:23:13.426876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.426848 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z"] Jul 24 14:23:13.433839 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.433813 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-676e8e92-7f4649f748-4vf6z"] Jul 24 14:23:13.749659 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.749578 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be5c842-f51a-4696-8012-71fb3a5b3594" path="/var/lib/kubelet/pods/0be5c842-f51a-4696-8012-71fb3a5b3594/volumes" Jul 24 14:23:13.749955 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:13.749942 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d9f8547-386f-4e7a-85db-d17833754a05" path="/var/lib/kubelet/pods/2d9f8547-386f-4e7a-85db-d17833754a05/volumes" Jul 24 14:23:17.449028 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:17.448970 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:18.902189 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:18.902156 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:20.418035 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:20.418005 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:20.498883 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:20.498854 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:20.864798 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:20.864766 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:25.918099 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:25.918065 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:23:33.526862 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:23:33.526824 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:01.999600 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:01.999560 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:03.281423 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:03.281389 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:08.733684 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:08.733603 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:08.837678 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:08.837638 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:09.046231 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:09.046197 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx"] Jul 24 14:24:09.046454 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:09.046429 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" containerID="cri-o://96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906" gracePeriod=30 Jul 24 14:24:09.051257 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:09.051225 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:09.294423 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:09.294392 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:10.471116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:10.471080 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:10.563190 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:10.563162 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:10.663820 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:10.663790 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:11.456624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:11.456583 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:11.554924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:11.554885 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:11.655144 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:11.655111 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:13.137269 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:13.137232 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:14.321047 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.320955 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:24:14.387671 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387636 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-credential-socket\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.387880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387728 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gdvw\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-kube-api-access-4gdvw\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.387880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387757 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-socket\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.387880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387795 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-podinfo\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.387880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387814 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-data\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.387880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387828 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-envoy\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.387880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387863 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-token\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.388250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387911 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istiod-ca-cert\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.388250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.387960 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-certs\") pod \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\" (UID: \"f7b3226d-366c-4e17-8ba1-bd06a2bf8050\") " Jul 24 14:24:14.388250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.388035 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:14.388409 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.388341 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.388409 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.388371 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:24:14.388514 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.388404 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:14.388587 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.388530 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:14.388973 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.388869 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:14.390436 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.390411 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:14.390556 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.390529 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:24:14.390773 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.390742 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:24:14.390773 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.390751 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-kube-api-access-4gdvw" (OuterVolumeSpecName: "kube-api-access-4gdvw") pod "f7b3226d-366c-4e17-8ba1-bd06a2bf8050" (UID: "f7b3226d-366c-4e17-8ba1-bd06a2bf8050"). InnerVolumeSpecName "kube-api-access-4gdvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489709 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489751 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489764 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489775 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gdvw\" (UniqueName: \"kubernetes.io/projected/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-kube-api-access-4gdvw\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489790 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489803 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489815 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.489875 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.489827 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7b3226d-366c-4e17-8ba1-bd06a2bf8050-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:24:14.685465 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.685363 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerID="96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906" exitCode=0 Jul 24 14:24:14.685465 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.685415 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" Jul 24 14:24:14.685465 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.685454 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" event={"ID":"f7b3226d-366c-4e17-8ba1-bd06a2bf8050","Type":"ContainerDied","Data":"96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906"} Jul 24 14:24:14.685720 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.685510 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx" event={"ID":"f7b3226d-366c-4e17-8ba1-bd06a2bf8050","Type":"ContainerDied","Data":"9e7d99dc734bc2b17c39e80af912719c7c6fe8ad124fb2b94cb0315c0edc34c2"} Jul 24 14:24:14.685720 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.685532 2555 scope.go:117] "RemoveContainer" containerID="96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906" Jul 24 14:24:14.695380 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.695359 2555 scope.go:117] "RemoveContainer" containerID="046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84" Jul 24 14:24:14.704470 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.704449 2555 scope.go:117] "RemoveContainer" containerID="96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906" Jul 24 14:24:14.704781 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:24:14.704761 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906\": container with ID starting with 96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906 not found: ID does not exist" containerID="96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906" Jul 24 14:24:14.704867 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.704789 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906"} err="failed to get container status \"96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906\": rpc error: code = NotFound desc = could not find container \"96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906\": container with ID starting with 96b48bfa89c6efbb5b3198df12fddaef56d6730ffce1712788bbc81004c58906 not found: ID does not exist" Jul 24 14:24:14.704867 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.704808 2555 scope.go:117] "RemoveContainer" containerID="046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84" Jul 24 14:24:14.705091 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:24:14.705066 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84\": container with ID starting with 046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84 not found: ID does not exist" containerID="046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84" Jul 24 14:24:14.705142 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.705100 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84"} err="failed to get container status \"046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84\": rpc error: code = NotFound desc = could not find container \"046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84\": container with ID starting with 046c286c032cf06d29339e71d4e9a02d1ec8ccaa16c65c9bfe712808a3b9ea84 not found: ID does not exist" Jul 24 14:24:14.709323 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.709284 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx"] Jul 24 14:24:14.713699 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:14.713665 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-676e8e92-openshift-default-5cb9d9f5b8-dp5wx"] Jul 24 14:24:15.749906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:15.749871 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" path="/var/lib/kubelet/pods/f7b3226d-366c-4e17-8ba1-bd06a2bf8050/volumes" Jul 24 14:24:18.888216 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:18.888158 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:18.956755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:18.956714 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:19.601308 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:19.601268 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:26.298314 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:26.298276 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:26.363966 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:26.363915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:26.470840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:26.470804 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:26.564200 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:26.564114 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:37.029978 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:37.029938 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:37.130431 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:37.130395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:37.452778 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:37.452745 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:24:58.470444 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:24:58.470409 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:05.274923 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:05.274888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:05.377558 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:05.377517 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:05.693857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:05.693817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:06.024322 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024231 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq"] Jul 24 14:25:06.024768 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024753 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" Jul 24 14:25:06.024818 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024771 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" Jul 24 14:25:06.024818 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024794 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d9f8547-386f-4e7a-85db-d17833754a05" containerName="payload-processing" Jul 24 14:25:06.024818 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024803 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d9f8547-386f-4e7a-85db-d17833754a05" containerName="payload-processing" Jul 24 14:25:06.024914 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024824 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be5c842-f51a-4696-8012-71fb3a5b3594" containerName="payload-pre-processing" Jul 24 14:25:06.024914 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024835 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be5c842-f51a-4696-8012-71fb3a5b3594" containerName="payload-pre-processing" Jul 24 14:25:06.024998 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024925 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be5c842-f51a-4696-8012-71fb3a5b3594" containerName="payload-pre-processing" Jul 24 14:25:06.024998 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024938 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d9f8547-386f-4e7a-85db-d17833754a05" containerName="payload-processing" Jul 24 14:25:06.024998 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024949 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" Jul 24 14:25:06.024998 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.024959 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" Jul 24 14:25:06.028335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.028315 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.031698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.031676 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bbfced-openshift-default-dockercfg-5nj4c\"" Jul 24 14:25:06.068032 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.067976 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq"] Jul 24 14:25:06.075321 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075197 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-envoy\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075321 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075256 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-data\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075321 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075286 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istiod-ca-cert\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075617 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075381 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-credential-socket\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075617 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075427 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-podinfo\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075617 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075486 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-certs\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075617 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075527 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-socket\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075617 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075584 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-token\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.075886 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.075618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99j8g\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-kube-api-access-99j8g\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.095853 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.095817 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:06.176286 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-certs\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176296 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-socket\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-token\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99j8g\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-kube-api-access-99j8g\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176657 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176509 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-envoy\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176657 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176579 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-data\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176657 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istiod-ca-cert\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176823 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176669 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-credential-socket\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176823 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176708 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-podinfo\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176823 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-certs\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.176823 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176743 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-socket\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.177049 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.176930 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-data\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.177124 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.177104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-credential-socket\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.177371 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.177352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istiod-ca-cert\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.178998 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.178959 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-envoy\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.179402 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.179380 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-podinfo\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.190207 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.190178 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-token\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.190936 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.190912 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99j8g\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-kube-api-access-99j8g\") pod \"e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.327657 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.327627 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:06.341654 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.341622 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:06.494086 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.494055 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq"] Jul 24 14:25:06.496185 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:25:06.496155 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d92ef79_4b62_463b_af5c_d432d3c3e6f1.slice/crio-42703644f156c0f9516b147110e1d13de74d025fcec1bd55e83937c483c13375 WatchSource:0}: Error finding container 42703644f156c0f9516b147110e1d13de74d025fcec1bd55e83937c483c13375: Status 404 returned error can't find the container with id 42703644f156c0f9516b147110e1d13de74d025fcec1bd55e83937c483c13375 Jul 24 14:25:06.498250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.498219 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:06.498320 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.498289 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:06.498359 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.498318 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:06.962454 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.962403 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" event={"ID":"0d92ef79-4b62-463b-af5c-d432d3c3e6f1","Type":"ContainerStarted","Data":"9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3"} Jul 24 14:25:06.962643 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.962513 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" event={"ID":"0d92ef79-4b62-463b-af5c-d432d3c3e6f1","Type":"ContainerStarted","Data":"42703644f156c0f9516b147110e1d13de74d025fcec1bd55e83937c483c13375"} Jul 24 14:25:06.993371 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:06.993316 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podStartSLOduration=1.993298539 podStartE2EDuration="1.993298539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:06.991911292 +0000 UTC m=+2393.787254001" watchObservedRunningTime="2026-07-24 14:25:06.993298539 +0000 UTC m=+2393.788641269" Jul 24 14:25:07.342374 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:07.342338 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:07.347455 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:07.347426 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:07.967681 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:07.967647 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:07.968655 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:07.968638 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:09.078317 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:09.078270 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:09.231370 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:09.231340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:09.436849 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:09.436761 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:09.985824 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:09.985791 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:10.085561 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.085527 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:10.173804 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.173767 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss"] Jul 24 14:25:10.174170 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.174157 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" Jul 24 14:25:10.174215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.174172 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b3226d-366c-4e17-8ba1-bd06a2bf8050" containerName="istio-proxy" Jul 24 14:25:10.177364 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.177346 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.184311 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.184288 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-bbfced\"" Jul 24 14:25:10.184502 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.184483 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-bbfced-dockercfg-vnkhv\"" Jul 24 14:25:10.193065 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.193038 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss"] Jul 24 14:25:10.224949 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.224904 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf"] Jul 24 14:25:10.229158 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.229131 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.242912 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.242877 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf"] Jul 24 14:25:10.321648 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.321603 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qwzr\" (UniqueName: \"kubernetes.io/projected/aa8113a0-2596-4cf0-8da8-63156d34a9a1-kube-api-access-5qwzr\") pod \"payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.321845 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.321748 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d211446a-4e2c-4256-948c-f683a0764d06-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.321845 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.321804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl7mv\" (UniqueName: \"kubernetes.io/projected/d211446a-4e2c-4256-948c-f683a0764d06-kube-api-access-kl7mv\") pod \"payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.321845 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.321844 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa8113a0-2596-4cf0-8da8-63156d34a9a1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.422923 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.422873 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d211446a-4e2c-4256-948c-f683a0764d06-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.423178 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.422940 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7mv\" (UniqueName: \"kubernetes.io/projected/d211446a-4e2c-4256-948c-f683a0764d06-kube-api-access-kl7mv\") pod \"payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.423178 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.422971 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa8113a0-2596-4cf0-8da8-63156d34a9a1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.423178 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.423055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qwzr\" (UniqueName: \"kubernetes.io/projected/aa8113a0-2596-4cf0-8da8-63156d34a9a1-kube-api-access-5qwzr\") pod \"payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.423676 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.423657 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d211446a-4e2c-4256-948c-f683a0764d06-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.423730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.423678 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa8113a0-2596-4cf0-8da8-63156d34a9a1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.440812 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.440782 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl7mv\" (UniqueName: \"kubernetes.io/projected/d211446a-4e2c-4256-948c-f683a0764d06-kube-api-access-kl7mv\") pod \"payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.442527 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.442504 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qwzr\" (UniqueName: \"kubernetes.io/projected/aa8113a0-2596-4cf0-8da8-63156d34a9a1-kube-api-access-5qwzr\") pod \"payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.490229 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.490132 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:10.542258 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.542229 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.659961 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.659929 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss"] Jul 24 14:25:10.661087 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:25:10.661057 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd211446a_4e2c_4256_948c_f683a0764d06.slice/crio-31ae1e4f7d794f93cba6b7d8f63bde808a38dc054aefde333b268a7cea3f3d89 WatchSource:0}: Error finding container 31ae1e4f7d794f93cba6b7d8f63bde808a38dc054aefde333b268a7cea3f3d89: Status 404 returned error can't find the container with id 31ae1e4f7d794f93cba6b7d8f63bde808a38dc054aefde333b268a7cea3f3d89 Jul 24 14:25:10.710321 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.710292 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf"] Jul 24 14:25:10.712616 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:25:10.712588 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8113a0_2596_4cf0_8da8_63156d34a9a1.slice/crio-72fb5dc516405aed4e6a08e4562a836cc919ff3c61cc8cafe826f7db7911f1c2 WatchSource:0}: Error finding container 72fb5dc516405aed4e6a08e4562a836cc919ff3c61cc8cafe826f7db7911f1c2: Status 404 returned error can't find the container with id 72fb5dc516405aed4e6a08e4562a836cc919ff3c61cc8cafe826f7db7911f1c2 Jul 24 14:25:10.986534 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.986499 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" event={"ID":"aa8113a0-2596-4cf0-8da8-63156d34a9a1","Type":"ContainerStarted","Data":"4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242"} Jul 24 14:25:10.986751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.986593 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" event={"ID":"aa8113a0-2596-4cf0-8da8-63156d34a9a1","Type":"ContainerStarted","Data":"72fb5dc516405aed4e6a08e4562a836cc919ff3c61cc8cafe826f7db7911f1c2"} Jul 24 14:25:10.986751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.986645 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:10.988345 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.988318 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" event={"ID":"d211446a-4e2c-4256-948c-f683a0764d06","Type":"ContainerStarted","Data":"41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a"} Jul 24 14:25:10.988496 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.988367 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" event={"ID":"d211446a-4e2c-4256-948c-f683a0764d06","Type":"ContainerStarted","Data":"31ae1e4f7d794f93cba6b7d8f63bde808a38dc054aefde333b268a7cea3f3d89"} Jul 24 14:25:10.988496 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:10.988461 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:11.014805 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:11.014697 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" podStartSLOduration=1.014679312 podStartE2EDuration="1.014679312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:11.012149652 +0000 UTC m=+2397.807492385" watchObservedRunningTime="2026-07-24 14:25:11.014679312 +0000 UTC m=+2397.810022041" Jul 24 14:25:13.555634 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:13.555595 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:13.922307 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:13.922222 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:14.314568 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.314504 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" podStartSLOduration=4.31448769 podStartE2EDuration="4.31448769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:11.037560984 +0000 UTC m=+2397.832903714" watchObservedRunningTime="2026-07-24 14:25:14.31448769 +0000 UTC m=+2401.109830420" Jul 24 14:25:14.316570 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.316543 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx"] Jul 24 14:25:14.321633 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.321615 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.331783 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:25:14.331746 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.331783 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:25:14.331764 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-inf-b-aa51de-openshift-default-dockercfg-khndj\" is forbidden: User \"system:node:ip-10-0-143-204.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-204.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-aa51de-openshift-default-dockercfg-khndj\"" type="*v1.Secret" Jul 24 14:25:14.340445 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.340413 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx"] Jul 24 14:25:14.381940 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.381903 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:14.463476 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463438 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-podinfo\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463497 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-socket\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463523 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-envoy\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463564 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cce737b9-a391-4bfd-9b73-24306aef08a6-istiod-ca-cert\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463581 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msl56\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-kube-api-access-msl56\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-data\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463612 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-token\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463628 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-credential-socket\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.463902 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.463734 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-certs\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.564960 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.564855 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-podinfo\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.564960 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.564941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-socket\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.564975 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-envoy\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cce737b9-a391-4bfd-9b73-24306aef08a6-istiod-ca-cert\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msl56\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-kube-api-access-msl56\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565086 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-data\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565124 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-token\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565149 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-credential-socket\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565226 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-certs\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565497 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565467 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-socket\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565531 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-certs\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565735 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-credential-socket\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.565903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.565803 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-data\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.566040 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.566019 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cce737b9-a391-4bfd-9b73-24306aef08a6-istiod-ca-cert\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.567517 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.567497 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-envoy\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.567758 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.567738 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-podinfo\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.579435 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.579400 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msl56\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-kube-api-access-msl56\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.595931 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.595899 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-token\") pod \"e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:14.712184 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:14.712145 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:15.633691 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.633633 2555 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" secret="" err="failed to sync secret cache: timed out waiting for the condition" Jul 24 14:25:15.634160 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.633739 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:15.763445 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.763411 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-aa51de-openshift-default-dockercfg-khndj\"" Jul 24 14:25:15.785051 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.785021 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx"] Jul 24 14:25:15.787737 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:25:15.787704 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce737b9_a391_4bfd_9b73_24306aef08a6.slice/crio-9c5780682d2d6370995829716621fb0c361318bf1f9c8501a27f5a9a0c045d14 WatchSource:0}: Error finding container 9c5780682d2d6370995829716621fb0c361318bf1f9c8501a27f5a9a0c045d14: Status 404 returned error can't find the container with id 9c5780682d2d6370995829716621fb0c361318bf1f9c8501a27f5a9a0c045d14 Jul 24 14:25:15.790684 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.790645 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:15.790757 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.790743 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:15.790801 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:15.790786 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:16.020926 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:16.020835 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" event={"ID":"cce737b9-a391-4bfd-9b73-24306aef08a6","Type":"ContainerStarted","Data":"983cfd9a9b5823490f8eb209fb11a197e2dee33f6330fb9b6568674d4e06ea3d"} Jul 24 14:25:16.020926 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:16.020888 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" event={"ID":"cce737b9-a391-4bfd-9b73-24306aef08a6","Type":"ContainerStarted","Data":"9c5780682d2d6370995829716621fb0c361318bf1f9c8501a27f5a9a0c045d14"} Jul 24 14:25:16.052779 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:16.052721 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podStartSLOduration=2.052701389 podStartE2EDuration="2.052701389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:16.049520032 +0000 UTC m=+2402.844862766" watchObservedRunningTime="2026-07-24 14:25:16.052701389 +0000 UTC m=+2402.848044118" Jul 24 14:25:16.635053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:16.635015 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:16.640234 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:16.640208 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:17.027500 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:17.027407 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:17.028481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:17.028457 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:18.066293 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.066254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:18.167808 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.167776 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:18.263829 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.263799 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:18.360538 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.360443 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:18.789952 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.789840 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:18.888088 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.888033 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:18.895225 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.895194 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd"] Jul 24 14:25:18.913000 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.912948 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:18.913247 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.913218 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd"] Jul 24 14:25:18.916125 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:18.916100 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-aa51de\"" Jul 24 14:25:19.014645 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.014599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773f08e0-5a86-4774-bc86-22c4a46c1463-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.015032 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.014958 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2sf\" (UniqueName: \"kubernetes.io/projected/773f08e0-5a86-4774-bc86-22c4a46c1463-kube-api-access-jv2sf\") pod \"payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.116098 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.116061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2sf\" (UniqueName: \"kubernetes.io/projected/773f08e0-5a86-4774-bc86-22c4a46c1463-kube-api-access-jv2sf\") pod \"payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.116519 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.116150 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773f08e0-5a86-4774-bc86-22c4a46c1463-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.117286 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.116806 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773f08e0-5a86-4774-bc86-22c4a46c1463-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.126531 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.126503 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2sf\" (UniqueName: \"kubernetes.io/projected/773f08e0-5a86-4774-bc86-22c4a46c1463-kube-api-access-jv2sf\") pod \"payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.227596 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.227565 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:19.416025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:19.415890 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd"] Jul 24 14:25:19.419261 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:25:19.419220 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod773f08e0_5a86_4774_bc86_22c4a46c1463.slice/crio-8c647f3271c112f139d91193250c19b40d64fab50afc14aa69037837357b6592 WatchSource:0}: Error finding container 8c647f3271c112f139d91193250c19b40d64fab50afc14aa69037837357b6592: Status 404 returned error can't find the container with id 8c647f3271c112f139d91193250c19b40d64fab50afc14aa69037837357b6592 Jul 24 14:25:20.045678 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:20.045639 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" event={"ID":"773f08e0-5a86-4774-bc86-22c4a46c1463","Type":"ContainerStarted","Data":"c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238"} Jul 24 14:25:20.045844 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:20.045703 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" event={"ID":"773f08e0-5a86-4774-bc86-22c4a46c1463","Type":"ContainerStarted","Data":"8c647f3271c112f139d91193250c19b40d64fab50afc14aa69037837357b6592"} Jul 24 14:25:20.045844 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:20.045726 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:20.065286 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:20.065224 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" podStartSLOduration=2.065205591 podStartE2EDuration="2.065205591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:25:20.063500962 +0000 UTC m=+2406.858843740" watchObservedRunningTime="2026-07-24 14:25:20.065205591 +0000 UTC m=+2406.860548356" Jul 24 14:25:21.999895 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:21.999847 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:25:22.001001 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:22.000953 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:25:22.620235 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:22.620204 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:24.010170 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:24.010130 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 14:25:24.010573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:24.010188 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 14:25:26.080436 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:26.080383 2555 generic.go:356] "Generic (PLEG): container finished" podID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerID="9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3" exitCode=0 Jul 24 14:25:26.080880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:26.080447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" event={"ID":"0d92ef79-4b62-463b-af5c-d432d3c3e6f1","Type":"ContainerDied","Data":"9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3"} Jul 24 14:25:26.080933 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:26.080921 2555 scope.go:117] "RemoveContainer" containerID="9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3" Jul 24 14:25:26.086033 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:26.082135 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:26.086033 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:26.082228 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:26.086033 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:26.082275 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:27.091413 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:27.091368 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" event={"ID":"0d92ef79-4b62-463b-af5c-d432d3c3e6f1","Type":"ContainerStarted","Data":"2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2"} Jul 24 14:25:27.342855 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:27.342755 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:27.343059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:27.342865 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 14:25:27.343059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:27.342903 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 14:25:28.342272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:28.342240 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 14:25:28.342777 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:28.342749 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 14:25:29.342552 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:29.342510 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 14:25:29.342944 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:29.342583 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 14:25:29.999685 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:29.999623 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:25:30.342447 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:30.342399 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 14:25:30.342670 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:30.342456 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 14:25:31.052730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:31.052701 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:25:31.342426 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:31.342330 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 14:25:31.342426 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:31.342389 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 14:25:32.346721 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:32.346684 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:32.347150 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:32.347113 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:32.348017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:32.347962 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:25:32.725837 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:32.725495 2555 patch_prober.go:28] interesting pod/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": read tcp 10.133.0.2:52380->10.133.0.69:15021: read: connection reset by peer" start-of-body= Jul 24 14:25:32.725837 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:32.725565 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": read tcp 10.133.0.2:52380->10.133.0.69:15021: read: connection reset by peer" Jul 24 14:25:33.134171 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:33.134136 2555 generic.go:356] "Generic (PLEG): container finished" podID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerID="983cfd9a9b5823490f8eb209fb11a197e2dee33f6330fb9b6568674d4e06ea3d" exitCode=0 Jul 24 14:25:33.134380 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:33.134207 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" event={"ID":"cce737b9-a391-4bfd-9b73-24306aef08a6","Type":"ContainerDied","Data":"983cfd9a9b5823490f8eb209fb11a197e2dee33f6330fb9b6568674d4e06ea3d"} Jul 24 14:25:33.134766 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:33.134749 2555 scope.go:117] "RemoveContainer" containerID="983cfd9a9b5823490f8eb209fb11a197e2dee33f6330fb9b6568674d4e06ea3d" Jul 24 14:25:33.135785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:33.135756 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:33.135882 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:33.135848 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:33.135927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:33.135884 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:25:34.141019 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:34.140918 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" event={"ID":"cce737b9-a391-4bfd-9b73-24306aef08a6","Type":"ContainerStarted","Data":"5b65bd3440bb834eb806c2dafbdf5e67ecb3fcd3d82b3c0f7359ec6c17a2eebf"} Jul 24 14:25:34.634670 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:34.634622 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:34.634960 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:34.634935 2555 patch_prober.go:28] interesting pod/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 24 14:25:34.635060 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:34.635024 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 24 14:25:35.634807 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:35.634764 2555 patch_prober.go:28] interesting pod/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 24 14:25:35.635248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:35.634834 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 24 14:25:36.634492 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:36.634448 2555 patch_prober.go:28] interesting pod/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 24 14:25:36.634685 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:36.634511 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 24 14:25:37.634780 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:37.634741 2555 patch_prober.go:28] interesting pod/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 24 14:25:37.635310 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:37.634810 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 24 14:25:38.638149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:38.638116 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:38.638534 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:38.638494 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:25:38.639248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:25:38.639231 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:26:09.315504 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:26:09.315465 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:26:52.677396 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:26:52.677342 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:26:53.807579 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:26:53.807541 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:04.213778 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.213694 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:04.233594 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.233559 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf"] Jul 24 14:27:04.233863 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.233831 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" podUID="aa8113a0-2596-4cf0-8da8-63156d34a9a1" containerName="payload-processing" containerID="cri-o://4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242" gracePeriod=30 Jul 24 14:27:04.243653 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.243081 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss"] Jul 24 14:27:04.243653 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.243328 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" podUID="d211446a-4e2c-4256-948c-f683a0764d06" containerName="payload-pre-processing" containerID="cri-o://41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a" gracePeriod=30 Jul 24 14:27:04.314181 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.314128 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:04.378783 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.378743 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:04.514922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.514887 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:27:04.551758 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.551731 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:27:04.632756 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.632717 2555 generic.go:356] "Generic (PLEG): container finished" podID="aa8113a0-2596-4cf0-8da8-63156d34a9a1" containerID="4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242" exitCode=0 Jul 24 14:27:04.632929 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.632782 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" Jul 24 14:27:04.632929 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.632809 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" event={"ID":"aa8113a0-2596-4cf0-8da8-63156d34a9a1","Type":"ContainerDied","Data":"4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242"} Jul 24 14:27:04.632929 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.632902 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf" event={"ID":"aa8113a0-2596-4cf0-8da8-63156d34a9a1","Type":"ContainerDied","Data":"72fb5dc516405aed4e6a08e4562a836cc919ff3c61cc8cafe826f7db7911f1c2"} Jul 24 14:27:04.633084 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.632935 2555 scope.go:117] "RemoveContainer" containerID="4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242" Jul 24 14:27:04.634291 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.634270 2555 generic.go:356] "Generic (PLEG): container finished" podID="d211446a-4e2c-4256-948c-f683a0764d06" containerID="41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a" exitCode=0 Jul 24 14:27:04.634363 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.634304 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" event={"ID":"d211446a-4e2c-4256-948c-f683a0764d06","Type":"ContainerDied","Data":"41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a"} Jul 24 14:27:04.634363 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.634329 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" Jul 24 14:27:04.634363 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.634350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss" event={"ID":"d211446a-4e2c-4256-948c-f683a0764d06","Type":"ContainerDied","Data":"31ae1e4f7d794f93cba6b7d8f63bde808a38dc054aefde333b268a7cea3f3d89"} Jul 24 14:27:04.642317 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.642295 2555 scope.go:117] "RemoveContainer" containerID="4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242" Jul 24 14:27:04.642666 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:27:04.642644 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242\": container with ID starting with 4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242 not found: ID does not exist" containerID="4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242" Jul 24 14:27:04.642711 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.642676 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242"} err="failed to get container status \"4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242\": rpc error: code = NotFound desc = could not find container \"4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242\": container with ID starting with 4515961aa82d7f6c4a9dae9192d6209dd6cf527c8f5393bc7e48ab071e53d242 not found: ID does not exist" Jul 24 14:27:04.642711 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.642692 2555 scope.go:117] "RemoveContainer" containerID="41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a" Jul 24 14:27:04.651759 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.651735 2555 scope.go:117] "RemoveContainer" containerID="41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a" Jul 24 14:27:04.652148 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:27:04.652126 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a\": container with ID starting with 41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a not found: ID does not exist" containerID="41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a" Jul 24 14:27:04.652228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.652160 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a"} err="failed to get container status \"41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a\": rpc error: code = NotFound desc = could not find container \"41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a\": container with ID starting with 41d4284fb31c99402b5f33245e48df40d9bd001900fd098dea3ce1190a6ece0a not found: ID does not exist" Jul 24 14:27:04.671515 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.671479 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qwzr\" (UniqueName: \"kubernetes.io/projected/aa8113a0-2596-4cf0-8da8-63156d34a9a1-kube-api-access-5qwzr\") pod \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " Jul 24 14:27:04.671708 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.671567 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d211446a-4e2c-4256-948c-f683a0764d06-plugins-config-volume\") pod \"d211446a-4e2c-4256-948c-f683a0764d06\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " Jul 24 14:27:04.671708 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.671689 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa8113a0-2596-4cf0-8da8-63156d34a9a1-plugins-config-volume\") pod \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\" (UID: \"aa8113a0-2596-4cf0-8da8-63156d34a9a1\") " Jul 24 14:27:04.671836 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.671716 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kl7mv\" (UniqueName: \"kubernetes.io/projected/d211446a-4e2c-4256-948c-f683a0764d06-kube-api-access-kl7mv\") pod \"d211446a-4e2c-4256-948c-f683a0764d06\" (UID: \"d211446a-4e2c-4256-948c-f683a0764d06\") " Jul 24 14:27:04.672090 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.672057 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d211446a-4e2c-4256-948c-f683a0764d06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d211446a-4e2c-4256-948c-f683a0764d06" (UID: "d211446a-4e2c-4256-948c-f683a0764d06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:27:04.672226 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.672090 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa8113a0-2596-4cf0-8da8-63156d34a9a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa8113a0-2596-4cf0-8da8-63156d34a9a1" (UID: "aa8113a0-2596-4cf0-8da8-63156d34a9a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:27:04.673924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.673902 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa8113a0-2596-4cf0-8da8-63156d34a9a1-kube-api-access-5qwzr" (OuterVolumeSpecName: "kube-api-access-5qwzr") pod "aa8113a0-2596-4cf0-8da8-63156d34a9a1" (UID: "aa8113a0-2596-4cf0-8da8-63156d34a9a1"). InnerVolumeSpecName "kube-api-access-5qwzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:04.674053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.673940 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d211446a-4e2c-4256-948c-f683a0764d06-kube-api-access-kl7mv" (OuterVolumeSpecName: "kube-api-access-kl7mv") pod "d211446a-4e2c-4256-948c-f683a0764d06" (UID: "d211446a-4e2c-4256-948c-f683a0764d06"). InnerVolumeSpecName "kube-api-access-kl7mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:04.773573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.773474 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa8113a0-2596-4cf0-8da8-63156d34a9a1-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:04.773573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.773511 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl7mv\" (UniqueName: \"kubernetes.io/projected/d211446a-4e2c-4256-948c-f683a0764d06-kube-api-access-kl7mv\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:04.773573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.773520 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qwzr\" (UniqueName: \"kubernetes.io/projected/aa8113a0-2596-4cf0-8da8-63156d34a9a1-kube-api-access-5qwzr\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:04.773573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.773529 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d211446a-4e2c-4256-948c-f683a0764d06-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:04.957165 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.957126 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf"] Jul 24 14:27:04.960809 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.960776 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-bbfced-66f6b5cd6-w4kcf"] Jul 24 14:27:04.972749 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.972708 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss"] Jul 24 14:27:04.974655 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:04.974627 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-bbfced-66d69df748-ghzss"] Jul 24 14:27:05.749556 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:05.749509 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa8113a0-2596-4cf0-8da8-63156d34a9a1" path="/var/lib/kubelet/pods/aa8113a0-2596-4cf0-8da8-63156d34a9a1/volumes" Jul 24 14:27:05.750105 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:05.750086 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d211446a-4e2c-4256-948c-f683a0764d06" path="/var/lib/kubelet/pods/d211446a-4e2c-4256-948c-f683a0764d06/volumes" Jul 24 14:27:08.804699 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:08.804669 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:09.229802 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:09.229710 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:15.071568 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:15.071528 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:15.171513 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:15.171484 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:15.337236 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:15.337136 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq"] Jul 24 14:27:15.337424 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:15.337399 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" containerID="cri-o://2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2" gracePeriod=30 Jul 24 14:27:15.361215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:15.361182 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:15.607162 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:15.607069 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:17.349310 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:17.349270 2555 patch_prober.go:28] interesting pod/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:27:17.349702 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:17.349345 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:27:20.608832 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.608750 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:27:20.624738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.624701 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-credential-socket\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.624738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.624744 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-token\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.624780 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istiod-ca-cert\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.624866 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-socket\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.624929 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:20.625021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.624956 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-podinfo\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625021 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625019 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99j8g\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-kube-api-access-99j8g\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625279 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625066 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-envoy\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625279 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625109 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-certs\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625279 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625151 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:20.625279 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625199 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-data\") pod \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\" (UID: \"0d92ef79-4b62-463b-af5c-d432d3c3e6f1\") " Jul 24 14:27:20.625279 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625200 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:27:20.625522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625340 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:20.625830 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625785 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.625830 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625808 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.625830 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625825 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.626081 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.625840 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.626394 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.626371 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:20.627348 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.627322 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:20.627480 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.627455 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-kube-api-access-99j8g" (OuterVolumeSpecName: "kube-api-access-99j8g") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "kube-api-access-99j8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:20.627651 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.627620 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:20.627651 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.627630 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d92ef79-4b62-463b-af5c-d432d3c3e6f1" (UID: "0d92ef79-4b62-463b-af5c-d432d3c3e6f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:27:20.723738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.723706 2555 generic.go:356] "Generic (PLEG): container finished" podID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerID="2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2" exitCode=0 Jul 24 14:27:20.723927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.723755 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" event={"ID":"0d92ef79-4b62-463b-af5c-d432d3c3e6f1","Type":"ContainerDied","Data":"2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2"} Jul 24 14:27:20.723927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.723779 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" Jul 24 14:27:20.723927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.723799 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq" event={"ID":"0d92ef79-4b62-463b-af5c-d432d3c3e6f1","Type":"ContainerDied","Data":"42703644f156c0f9516b147110e1d13de74d025fcec1bd55e83937c483c13375"} Jul 24 14:27:20.723927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.723821 2555 scope.go:117] "RemoveContainer" containerID="2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2" Jul 24 14:27:20.726640 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.726615 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.726640 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.726640 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.726840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.726651 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.726840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.726665 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99j8g\" (UniqueName: \"kubernetes.io/projected/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-kube-api-access-99j8g\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.726840 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.726674 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d92ef79-4b62-463b-af5c-d432d3c3e6f1-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:20.733675 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.733655 2555 scope.go:117] "RemoveContainer" containerID="9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3" Jul 24 14:27:20.742480 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.742455 2555 scope.go:117] "RemoveContainer" containerID="2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2" Jul 24 14:27:20.742822 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:27:20.742801 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2\": container with ID starting with 2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2 not found: ID does not exist" containerID="2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2" Jul 24 14:27:20.742866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.742833 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2"} err="failed to get container status \"2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2\": rpc error: code = NotFound desc = could not find container \"2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2\": container with ID starting with 2ceebe61427cad958f2a99f148f0b5f642917446c68df06e92b60373aae001f2 not found: ID does not exist" Jul 24 14:27:20.742866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.742856 2555 scope.go:117] "RemoveContainer" containerID="9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3" Jul 24 14:27:20.743162 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:27:20.743142 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3\": container with ID starting with 9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3 not found: ID does not exist" containerID="9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3" Jul 24 14:27:20.743206 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.743169 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3"} err="failed to get container status \"9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3\": rpc error: code = NotFound desc = could not find container \"9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3\": container with ID starting with 9c89059354ae029ce5a9e46920530686b2ef3e328049a03f89b828bddac45fe3 not found: ID does not exist" Jul 24 14:27:20.746809 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.746779 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq"] Jul 24 14:27:20.750790 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:20.750755 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bbfced-openshift-default-696cfbf7db-stjdq"] Jul 24 14:27:21.750565 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:21.750524 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" path="/var/lib/kubelet/pods/0d92ef79-4b62-463b-af5c-d432d3c3e6f1/volumes" Jul 24 14:27:26.165591 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.165558 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:26.204358 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.204321 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd"] Jul 24 14:27:26.204607 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.204570 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" podUID="773f08e0-5a86-4774-bc86-22c4a46c1463" containerName="payload-pre-processing" containerID="cri-o://c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238" gracePeriod=30 Jul 24 14:27:26.266070 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.266041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:26.369501 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.369466 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:26.462959 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.462932 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:27:26.582368 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.582327 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773f08e0-5a86-4774-bc86-22c4a46c1463-plugins-config-volume\") pod \"773f08e0-5a86-4774-bc86-22c4a46c1463\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " Jul 24 14:27:26.582549 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.582444 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv2sf\" (UniqueName: \"kubernetes.io/projected/773f08e0-5a86-4774-bc86-22c4a46c1463-kube-api-access-jv2sf\") pod \"773f08e0-5a86-4774-bc86-22c4a46c1463\" (UID: \"773f08e0-5a86-4774-bc86-22c4a46c1463\") " Jul 24 14:27:26.582786 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.582763 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/773f08e0-5a86-4774-bc86-22c4a46c1463-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "773f08e0-5a86-4774-bc86-22c4a46c1463" (UID: "773f08e0-5a86-4774-bc86-22c4a46c1463"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:27:26.584959 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.584930 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/773f08e0-5a86-4774-bc86-22c4a46c1463-kube-api-access-jv2sf" (OuterVolumeSpecName: "kube-api-access-jv2sf") pod "773f08e0-5a86-4774-bc86-22c4a46c1463" (UID: "773f08e0-5a86-4774-bc86-22c4a46c1463"). InnerVolumeSpecName "kube-api-access-jv2sf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:26.683734 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.683693 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/773f08e0-5a86-4774-bc86-22c4a46c1463-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:26.683734 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.683736 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv2sf\" (UniqueName: \"kubernetes.io/projected/773f08e0-5a86-4774-bc86-22c4a46c1463-kube-api-access-jv2sf\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:26.755945 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.755842 2555 generic.go:356] "Generic (PLEG): container finished" podID="773f08e0-5a86-4774-bc86-22c4a46c1463" containerID="c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238" exitCode=0 Jul 24 14:27:26.755945 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.755893 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" Jul 24 14:27:26.755945 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.755936 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" event={"ID":"773f08e0-5a86-4774-bc86-22c4a46c1463","Type":"ContainerDied","Data":"c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238"} Jul 24 14:27:26.756237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.756013 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd" event={"ID":"773f08e0-5a86-4774-bc86-22c4a46c1463","Type":"ContainerDied","Data":"8c647f3271c112f139d91193250c19b40d64fab50afc14aa69037837357b6592"} Jul 24 14:27:26.756237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.756040 2555 scope.go:117] "RemoveContainer" containerID="c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238" Jul 24 14:27:26.765885 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.765867 2555 scope.go:117] "RemoveContainer" containerID="c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238" Jul 24 14:27:26.766225 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:27:26.766205 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238\": container with ID starting with c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238 not found: ID does not exist" containerID="c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238" Jul 24 14:27:26.766353 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.766236 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238"} err="failed to get container status \"c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238\": rpc error: code = NotFound desc = could not find container \"c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238\": container with ID starting with c6eaeea36da9837873374864f7fd85f15b4f6bb40e7c61620061d98df0e0e238 not found: ID does not exist" Jul 24 14:27:26.779230 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.779199 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd"] Jul 24 14:27:26.782796 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:26.782769 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aa51de-8469595848-b7pjd"] Jul 24 14:27:27.749801 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:27.749766 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="773f08e0-5a86-4774-bc86-22c4a46c1463" path="/var/lib/kubelet/pods/773f08e0-5a86-4774-bc86-22c4a46c1463/volumes" Jul 24 14:27:30.801853 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:30.801816 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:31.209305 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:31.209196 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:37.431880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:37.431842 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:37.498153 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:37.498122 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:37.720478 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:37.720378 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx"] Jul 24 14:27:37.720679 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:37.720652 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" containerID="cri-o://5b65bd3440bb834eb806c2dafbdf5e67ecb3fcd3d82b3c0f7359ec6c17a2eebf" gracePeriod=30 Jul 24 14:27:37.742957 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:37.742924 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:37.971155 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:37.971064 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:38.374626 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.374589 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:38.641933 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.641840 2555 patch_prober.go:28] interesting pod/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:27:38.641933 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.641914 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:27:38.661621 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.661585 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w"] Jul 24 14:27:38.661956 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.661945 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" Jul 24 14:27:38.662016 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.661958 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" Jul 24 14:27:38.662016 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.661966 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" Jul 24 14:27:38.662016 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.661971 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662021 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa8113a0-2596-4cf0-8da8-63156d34a9a1" containerName="payload-processing" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662028 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa8113a0-2596-4cf0-8da8-63156d34a9a1" containerName="payload-processing" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662035 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="773f08e0-5a86-4774-bc86-22c4a46c1463" containerName="payload-pre-processing" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662040 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="773f08e0-5a86-4774-bc86-22c4a46c1463" containerName="payload-pre-processing" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662051 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d211446a-4e2c-4256-948c-f683a0764d06" containerName="payload-pre-processing" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662056 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d211446a-4e2c-4256-948c-f683a0764d06" containerName="payload-pre-processing" Jul 24 14:27:38.662116 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662118 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" Jul 24 14:27:38.662351 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662126 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="773f08e0-5a86-4774-bc86-22c4a46c1463" containerName="payload-pre-processing" Jul 24 14:27:38.662351 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662133 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa8113a0-2596-4cf0-8da8-63156d34a9a1" containerName="payload-processing" Jul 24 14:27:38.662351 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.662140 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d211446a-4e2c-4256-948c-f683a0764d06" containerName="payload-pre-processing" Jul 24 14:27:38.665058 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.665041 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.668104 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.668082 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e7d231-openshift-default-dockercfg-449pz\"" Jul 24 14:27:38.678069 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.678044 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w"] Jul 24 14:27:38.693478 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.693450 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:38.781054 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.781023 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:38.802859 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.802822 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a30e88-43ed-4049-9619-f744670c1fdb-istio-podinfo\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.802859 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.802864 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-credential-socket\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803113 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.802927 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-socket\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803113 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.802964 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-envoy\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803113 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.803064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a30e88-43ed-4049-9619-f744670c1fdb-istiod-ca-cert\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803113 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.803090 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-data\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803113 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.803108 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-certs\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803354 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.803125 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnchq\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-kube-api-access-pnchq\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.803354 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.803229 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-istio-token\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.904751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904645 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a30e88-43ed-4049-9619-f744670c1fdb-istiod-ca-cert\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.904751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904700 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-data\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.904751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904726 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-certs\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.904751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904752 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnchq\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-kube-api-access-pnchq\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904788 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-istio-token\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904831 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a30e88-43ed-4049-9619-f744670c1fdb-istio-podinfo\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-credential-socket\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904914 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-socket\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.904941 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-envoy\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905424 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.905221 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-data\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905485 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.905422 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-certs\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905485 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.905422 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-credential-socket\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905485 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.905470 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-socket\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.905600 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.905500 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a30e88-43ed-4049-9619-f744670c1fdb-istiod-ca-cert\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.907404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.907376 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-envoy\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.907616 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.907589 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a30e88-43ed-4049-9619-f744670c1fdb-istio-podinfo\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.913929 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.913902 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-istio-token\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.914139 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.914119 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnchq\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-kube-api-access-pnchq\") pod \"e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:38.976261 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:38.976221 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:39.137781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.137592 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w"] Jul 24 14:27:39.140570 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:27:39.140536 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a30e88_43ed_4049_9619_f744670c1fdb.slice/crio-90f1c13d6885dec690dd1b526254547ebafca5e41ad315e96e354a71d87c9324 WatchSource:0}: Error finding container 90f1c13d6885dec690dd1b526254547ebafca5e41ad315e96e354a71d87c9324: Status 404 returned error can't find the container with id 90f1c13d6885dec690dd1b526254547ebafca5e41ad315e96e354a71d87c9324 Jul 24 14:27:39.142909 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.142864 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:39.143014 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.142960 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:39.143061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.143037 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:39.826778 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.826743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" event={"ID":"27a30e88-43ed-4049-9619-f744670c1fdb","Type":"ContainerStarted","Data":"eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa"} Jul 24 14:27:39.827267 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.826803 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" event={"ID":"27a30e88-43ed-4049-9619-f744670c1fdb","Type":"ContainerStarted","Data":"90f1c13d6885dec690dd1b526254547ebafca5e41ad315e96e354a71d87c9324"} Jul 24 14:27:39.853055 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.852999 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podStartSLOduration=1.852957304 podStartE2EDuration="1.852957304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:27:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:27:39.850652078 +0000 UTC m=+2546.645994805" watchObservedRunningTime="2026-07-24 14:27:39.852957304 +0000 UTC m=+2546.648300033" Jul 24 14:27:39.976673 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:39.976639 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:40.982777 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:40.982737 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:41.837272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:41.837243 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:41.838170 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:41.838150 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:41.895996 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:41.895942 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:41.989460 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:41.989421 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:42.108000 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.107889 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:42.216193 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.216139 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:42.596216 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.596181 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:42.698316 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.698286 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:42.844804 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.844761 2555 generic.go:356] "Generic (PLEG): container finished" podID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerID="5b65bd3440bb834eb806c2dafbdf5e67ecb3fcd3d82b3c0f7359ec6c17a2eebf" exitCode=0 Jul 24 14:27:42.845035 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.844805 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" event={"ID":"cce737b9-a391-4bfd-9b73-24306aef08a6","Type":"ContainerDied","Data":"5b65bd3440bb834eb806c2dafbdf5e67ecb3fcd3d82b3c0f7359ec6c17a2eebf"} Jul 24 14:27:42.845035 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.844873 2555 scope.go:117] "RemoveContainer" containerID="983cfd9a9b5823490f8eb209fb11a197e2dee33f6330fb9b6568674d4e06ea3d" Jul 24 14:27:42.984350 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:42.984316 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:27:43.047426 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047389 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-socket\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047426 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047430 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-certs\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047500 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-podinfo\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047522 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cce737b9-a391-4bfd-9b73-24306aef08a6-istiod-ca-cert\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047551 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-data\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047605 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-token\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047660 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047752 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-credential-socket\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047792 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-envoy\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047812 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:43.047974 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.047819 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msl56\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-kube-api-access-msl56\") pod \"cce737b9-a391-4bfd-9b73-24306aef08a6\" (UID: \"cce737b9-a391-4bfd-9b73-24306aef08a6\") " Jul 24 14:27:43.048662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.048031 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:43.048662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.048306 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.048662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.048331 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.048662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.048347 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.048662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.048376 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cce737b9-a391-4bfd-9b73-24306aef08a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:27:43.048662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.048532 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:43.050118 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.050090 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:27:43.050377 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.050355 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:43.050542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.050526 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-kube-api-access-msl56" (OuterVolumeSpecName: "kube-api-access-msl56") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "kube-api-access-msl56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:27:43.050587 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.050561 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cce737b9-a391-4bfd-9b73-24306aef08a6" (UID: "cce737b9-a391-4bfd-9b73-24306aef08a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:27:43.149522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.149425 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.149522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.149457 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cce737b9-a391-4bfd-9b73-24306aef08a6-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.149522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.149468 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.149522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.149476 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.149522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.149483 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cce737b9-a391-4bfd-9b73-24306aef08a6-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.149522 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.149492 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msl56\" (UniqueName: \"kubernetes.io/projected/cce737b9-a391-4bfd-9b73-24306aef08a6-kube-api-access-msl56\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:27:43.851969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.851922 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" Jul 24 14:27:43.851969 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.851967 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx" event={"ID":"cce737b9-a391-4bfd-9b73-24306aef08a6","Type":"ContainerDied","Data":"9c5780682d2d6370995829716621fb0c361318bf1f9c8501a27f5a9a0c045d14"} Jul 24 14:27:43.852242 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.852054 2555 scope.go:117] "RemoveContainer" containerID="5b65bd3440bb834eb806c2dafbdf5e67ecb3fcd3d82b3c0f7359ec6c17a2eebf" Jul 24 14:27:43.871453 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.871415 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx"] Jul 24 14:27:43.874958 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:43.874929 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-aa51de-openshift-default-57c8ddff94-z4rhx"] Jul 24 14:27:45.749252 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:45.749221 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" path="/var/lib/kubelet/pods/cce737b9-a391-4bfd-9b73-24306aef08a6/volumes" Jul 24 14:27:46.448797 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:46.448761 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:46.777133 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:46.777043 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:46.846451 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:46.846390 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:46.952374 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:46.952340 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:47.278044 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.278004 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8"] Jul 24 14:27:47.278538 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.278524 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" Jul 24 14:27:47.278601 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.278541 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" Jul 24 14:27:47.278660 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.278649 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d92ef79-4b62-463b-af5c-d432d3c3e6f1" containerName="istio-proxy" Jul 24 14:27:47.278709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.278664 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" Jul 24 14:27:47.278709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.278673 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" Jul 24 14:27:47.282046 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.282027 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.285174 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.285151 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-45b6b7-openshift-default-dockercfg-wz77n\"" Jul 24 14:27:47.295553 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.295528 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8"] Jul 24 14:27:47.311886 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.311851 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:47.391130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-data\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391129 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-socket\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391170 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-podinfo\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391216 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-credential-socket\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391240 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-certs\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391284 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istiod-ca-cert\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-token\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391362 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-envoy\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.391404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.391397 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wkk9\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-kube-api-access-2wkk9\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492616 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492581 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-token\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492616 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-envoy\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wkk9\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-kube-api-access-2wkk9\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492724 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-data\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492752 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-socket\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492787 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-podinfo\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492832 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-credential-socket\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.492876 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492858 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-certs\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.493204 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.492904 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istiod-ca-cert\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.493204 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.493104 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-data\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.493204 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.493191 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-socket\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.493360 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.493342 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-credential-socket\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.493415 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.493377 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-certs\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.493674 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.493653 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istiod-ca-cert\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.495164 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.495145 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-envoy\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.495700 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.495681 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-podinfo\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.501273 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.501244 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-token\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.501563 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.501540 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wkk9\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-kube-api-access-2wkk9\") pod \"e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.506730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.506704 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:47.593596 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.593553 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:47.747727 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:27:47.747694 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d28ae38_974b_40a7_8f1c_618449bc1a3a.slice/crio-cb02513dd8b9bde95d618c9da6cca6cb0f347db0f61871cbf4a023b00d2fb84e WatchSource:0}: Error finding container cb02513dd8b9bde95d618c9da6cca6cb0f347db0f61871cbf4a023b00d2fb84e: Status 404 returned error can't find the container with id cb02513dd8b9bde95d618c9da6cca6cb0f347db0f61871cbf4a023b00d2fb84e Jul 24 14:27:47.749866 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.749833 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:47.749954 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.749901 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:47.749954 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.749930 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:47.751059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.751001 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8"] Jul 24 14:27:47.875935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.875835 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" event={"ID":"4d28ae38-974b-40a7-8f1c-618449bc1a3a","Type":"ContainerStarted","Data":"baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df"} Jul 24 14:27:47.875935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.875890 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" event={"ID":"4d28ae38-974b-40a7-8f1c-618449bc1a3a","Type":"ContainerStarted","Data":"cb02513dd8b9bde95d618c9da6cca6cb0f347db0f61871cbf4a023b00d2fb84e"} Jul 24 14:27:47.896612 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:47.896526 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podStartSLOduration=0.896505344 podStartE2EDuration="896.505344ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:27:47.895535644 +0000 UTC m=+2554.690878396" watchObservedRunningTime="2026-07-24 14:27:47.896505344 +0000 UTC m=+2554.691848078" Jul 24 14:27:48.593949 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:48.593912 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:48.599166 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:48.599139 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:48.881105 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:48.881009 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:48.882393 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:48.882373 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:27:49.980745 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:49.980712 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:50.078190 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:50.078155 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:50.347571 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:50.347537 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:50.887365 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:50.887323 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:51.197053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.196932 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:51.271430 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.271396 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:51.272608 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.272588 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8"] Jul 24 14:27:51.273053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.273040 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" Jul 24 14:27:51.273094 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.273055 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce737b9-a391-4bfd-9b73-24306aef08a6" containerName="istio-proxy" Jul 24 14:27:51.275046 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.275031 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.279631 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.279610 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-45b6b7\"" Jul 24 14:27:51.290503 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.290463 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8"] Jul 24 14:27:51.434690 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.434641 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tw2c\" (UniqueName: \"kubernetes.io/projected/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-kube-api-access-9tw2c\") pod \"payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.434903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.434811 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.535852 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.535748 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.535852 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.535830 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tw2c\" (UniqueName: \"kubernetes.io/projected/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-kube-api-access-9tw2c\") pod \"payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.536536 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.536516 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.544928 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.544899 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tw2c\" (UniqueName: \"kubernetes.io/projected/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-kube-api-access-9tw2c\") pod \"payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.608561 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.608519 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:51.799007 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.796936 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8"] Jul 24 14:27:51.917237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.917188 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" event={"ID":"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3","Type":"ContainerStarted","Data":"207b4d4ff6548f1a52ac13fc666021161b44dd1d660855fae6bd52fb684e074d"} Jul 24 14:27:51.955601 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:51.955568 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:52.924709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:52.924660 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" event={"ID":"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3","Type":"ContainerStarted","Data":"1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd"} Jul 24 14:27:52.925132 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:52.924734 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:27:52.942566 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:52.942493 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" podStartSLOduration=1.94247368 podStartE2EDuration="1.94247368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:27:52.942248976 +0000 UTC m=+2559.737591710" watchObservedRunningTime="2026-07-24 14:27:52.94247368 +0000 UTC m=+2559.737816413" Jul 24 14:27:54.391935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:54.391894 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:27:57.360849 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.360818 2555 patch_prober.go:28] interesting pod/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": read tcp 10.133.0.2:56684->10.133.0.71:15021: read: connection reset by peer" start-of-body= Jul 24 14:27:57.361282 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.360884 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": read tcp 10.133.0.2:56684->10.133.0.71:15021: read: connection reset by peer" Jul 24 14:27:57.952479 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.952445 2555 generic.go:356] "Generic (PLEG): container finished" podID="27a30e88-43ed-4049-9619-f744670c1fdb" containerID="eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa" exitCode=0 Jul 24 14:27:57.952659 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.952523 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" event={"ID":"27a30e88-43ed-4049-9619-f744670c1fdb","Type":"ContainerDied","Data":"eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa"} Jul 24 14:27:57.952965 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.952951 2555 scope.go:117] "RemoveContainer" containerID="eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa" Jul 24 14:27:57.953992 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.953953 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:57.954066 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.954046 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:57.954103 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:57.954080 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:27:58.960780 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:58.960747 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" event={"ID":"27a30e88-43ed-4049-9619-f744670c1fdb","Type":"ContainerStarted","Data":"f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225"} Jul 24 14:27:58.977384 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:58.977344 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:27:59.977380 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:59.977342 2555 patch_prober.go:28] interesting pod/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 24 14:27:59.977783 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:27:59.977410 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 24 14:28:00.976626 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:00.976587 2555 patch_prober.go:28] interesting pod/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 24 14:28:00.976819 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:00.976641 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 24 14:28:01.977332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:01.977292 2555 patch_prober.go:28] interesting pod/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 24 14:28:01.977790 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:01.977349 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 24 14:28:02.976941 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:02.976903 2555 patch_prober.go:28] interesting pod/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 24 14:28:02.977154 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:02.976966 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 24 14:28:03.933060 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:03.933024 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:28:03.980917 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:03.980879 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:28:03.981269 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:03.981239 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:28:04.001592 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:04.001375 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:28:08.037910 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:08.037874 2555 generic.go:356] "Generic (PLEG): container finished" podID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerID="baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df" exitCode=0 Jul 24 14:28:08.038490 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:08.037922 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" event={"ID":"4d28ae38-974b-40a7-8f1c-618449bc1a3a","Type":"ContainerDied","Data":"baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df"} Jul 24 14:28:08.038567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:08.038486 2555 scope.go:117] "RemoveContainer" containerID="baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df" Jul 24 14:28:08.039593 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:08.039559 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:28:08.039687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:08.039632 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:28:08.039687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:08.039660 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:28:09.045198 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:09.045163 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" event={"ID":"4d28ae38-974b-40a7-8f1c-618449bc1a3a","Type":"ContainerStarted","Data":"0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9"} Jul 24 14:28:09.594626 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:09.594592 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:28:09.595167 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:09.595131 2555 patch_prober.go:28] interesting pod/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 24 14:28:09.595298 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:09.595203 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 24 14:28:10.594646 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:10.594184 2555 patch_prober.go:28] interesting pod/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 24 14:28:10.594646 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:10.594264 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 24 14:28:11.594932 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:11.594890 2555 patch_prober.go:28] interesting pod/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 24 14:28:11.595604 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:11.595574 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 24 14:28:12.594432 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:12.594395 2555 patch_prober.go:28] interesting pod/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 24 14:28:12.594755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:12.594714 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 24 14:28:13.598111 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:13.598082 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:28:13.598564 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:13.598447 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:28:13.599178 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:13.599148 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:28:17.872126 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:17.872097 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:17.980958 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:17.980930 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:18.082246 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:18.082207 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:22.903486 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:22.903444 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:23.366744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:23.366706 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:29.414193 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:29.414154 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:29.515250 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:29.515218 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:29.751191 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:29.751104 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w"] Jul 24 14:28:29.751410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:29.751383 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" containerID="cri-o://f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225" gracePeriod=30 Jul 24 14:28:29.833398 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:29.833360 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:30.027624 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:30.027539 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:33.984266 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:33.984169 2555 patch_prober.go:28] interesting pod/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:28:33.984266 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:33.984233 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:28:35.003233 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.003208 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:28:35.179052 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.178973 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-istio-token\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179069 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a30e88-43ed-4049-9619-f744670c1fdb-istio-podinfo\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179095 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-envoy\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179129 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-data\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179179 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a30e88-43ed-4049-9619-f744670c1fdb-istiod-ca-cert\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179228 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179219 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnchq\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-kube-api-access-pnchq\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179256 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-socket\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179331 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-credential-socket\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179328 2555 generic.go:356] "Generic (PLEG): container finished" podID="27a30e88-43ed-4049-9619-f744670c1fdb" containerID="f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225" exitCode=0 Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179385 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-certs\") pod \"27a30e88-43ed-4049-9619-f744670c1fdb\" (UID: \"27a30e88-43ed-4049-9619-f744670c1fdb\") " Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179470 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" event={"ID":"27a30e88-43ed-4049-9619-f744670c1fdb","Type":"ContainerDied","Data":"f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225"} Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179534 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" event={"ID":"27a30e88-43ed-4049-9619-f744670c1fdb","Type":"ContainerDied","Data":"90f1c13d6885dec690dd1b526254547ebafca5e41ad315e96e354a71d87c9324"} Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179566 2555 scope.go:117] "RemoveContainer" containerID="f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225" Jul 24 14:28:35.179738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179588 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27a30e88-43ed-4049-9619-f744670c1fdb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:28:35.180212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179747 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w" Jul 24 14:28:35.180212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179816 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:28:35.180212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.179864 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/27a30e88-43ed-4049-9619-f744670c1fdb-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.180212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.180144 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:28:35.180212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.180205 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:28:35.180542 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.180500 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-data" (OuterVolumeSpecName: "istio-data") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:28:35.181687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.181657 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/27a30e88-43ed-4049-9619-f744670c1fdb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:28:35.181832 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.181808 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-istio-token" (OuterVolumeSpecName: "istio-token") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:28:35.182240 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.182218 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:28:35.182397 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.182378 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-kube-api-access-pnchq" (OuterVolumeSpecName: "kube-api-access-pnchq") pod "27a30e88-43ed-4049-9619-f744670c1fdb" (UID: "27a30e88-43ed-4049-9619-f744670c1fdb"). InnerVolumeSpecName "kube-api-access-pnchq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:28:35.202011 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.201967 2555 scope.go:117] "RemoveContainer" containerID="eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa" Jul 24 14:28:35.210812 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.210791 2555 scope.go:117] "RemoveContainer" containerID="f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225" Jul 24 14:28:35.211217 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:28:35.211194 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225\": container with ID starting with f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225 not found: ID does not exist" containerID="f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225" Jul 24 14:28:35.211302 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.211225 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225"} err="failed to get container status \"f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225\": rpc error: code = NotFound desc = could not find container \"f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225\": container with ID starting with f55936b7559fe0eed47e002648758fe3853d312f2772dda1d7fa69a476525225 not found: ID does not exist" Jul 24 14:28:35.211302 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.211246 2555 scope.go:117] "RemoveContainer" containerID="eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa" Jul 24 14:28:35.211550 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:28:35.211527 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa\": container with ID starting with eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa not found: ID does not exist" containerID="eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa" Jul 24 14:28:35.211597 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.211557 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa"} err="failed to get container status \"eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa\": rpc error: code = NotFound desc = could not find container \"eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa\": container with ID starting with eca70a5c82e58f0c4fd43a3297777ec625195b6206ca4ed3f6ead3da645955fa not found: ID does not exist" Jul 24 14:28:35.280851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280814 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.280851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280841 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.280851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280850 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/27a30e88-43ed-4049-9619-f744670c1fdb-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.280851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280857 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.281129 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280865 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.281129 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280875 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnchq\" (UniqueName: \"kubernetes.io/projected/27a30e88-43ed-4049-9619-f744670c1fdb-kube-api-access-pnchq\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.281129 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280883 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.281129 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.280893 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/27a30e88-43ed-4049-9619-f744670c1fdb-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:35.504739 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.504706 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w"] Jul 24 14:28:35.509009 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.508966 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e7d231-openshift-default-7c5c8885c6-cwn9w"] Jul 24 14:28:35.750183 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:35.750084 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" path="/var/lib/kubelet/pods/27a30e88-43ed-4049-9619-f744670c1fdb/volumes" Jul 24 14:28:46.825519 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:46.825481 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:46.857492 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:46.857456 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8"] Jul 24 14:28:46.857721 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:46.857694 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" podUID="ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" containerName="payload-pre-processing" containerID="cri-o://1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd" gracePeriod=30 Jul 24 14:28:46.926541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:46.926506 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:47.034303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.034248 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:47.122324 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.122295 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:28:47.191008 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.190944 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tw2c\" (UniqueName: \"kubernetes.io/projected/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-kube-api-access-9tw2c\") pod \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " Jul 24 14:28:47.191219 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.191041 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-plugins-config-volume\") pod \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\" (UID: \"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3\") " Jul 24 14:28:47.191466 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.191434 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" (UID: "ab84f2b9-223b-4549-8ec6-941cf9e9b9c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:28:47.193288 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.193262 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-kube-api-access-9tw2c" (OuterVolumeSpecName: "kube-api-access-9tw2c") pod "ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" (UID: "ab84f2b9-223b-4549-8ec6-941cf9e9b9c3"). InnerVolumeSpecName "kube-api-access-9tw2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:28:47.243687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.243646 2555 generic.go:356] "Generic (PLEG): container finished" podID="ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" containerID="1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd" exitCode=0 Jul 24 14:28:47.243888 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.243696 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" Jul 24 14:28:47.243888 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.243739 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" event={"ID":"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3","Type":"ContainerDied","Data":"1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd"} Jul 24 14:28:47.243888 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.243825 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8" event={"ID":"ab84f2b9-223b-4549-8ec6-941cf9e9b9c3","Type":"ContainerDied","Data":"207b4d4ff6548f1a52ac13fc666021161b44dd1d660855fae6bd52fb684e074d"} Jul 24 14:28:47.243888 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.243848 2555 scope.go:117] "RemoveContainer" containerID="1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd" Jul 24 14:28:47.253364 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.253345 2555 scope.go:117] "RemoveContainer" containerID="1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd" Jul 24 14:28:47.253646 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:28:47.253627 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd\": container with ID starting with 1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd not found: ID does not exist" containerID="1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd" Jul 24 14:28:47.253689 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.253657 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd"} err="failed to get container status \"1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd\": rpc error: code = NotFound desc = could not find container \"1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd\": container with ID starting with 1d9646311140c7b961b21b637ed2ff9b9fe02abddc680600b789fa6ed87cd0bd not found: ID does not exist" Jul 24 14:28:47.266505 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.266473 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8"] Jul 24 14:28:47.273025 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.272997 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-45b6b7-665995c6dd-57mp8"] Jul 24 14:28:47.292792 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.292755 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tw2c\" (UniqueName: \"kubernetes.io/projected/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-kube-api-access-9tw2c\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:47.292792 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.292794 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:28:47.749813 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:47.749777 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" path="/var/lib/kubelet/pods/ab84f2b9-223b-4549-8ec6-941cf9e9b9c3/volumes" Jul 24 14:28:50.243674 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:50.243637 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:50.652037 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:50.651976 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:56.615290 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:56.615254 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:56.716276 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:56.716240 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:56.899541 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:56.899440 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8"] Jul 24 14:28:56.899736 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:56.899703 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" containerID="cri-o://0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9" gracePeriod=30 Jul 24 14:28:56.991286 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:56.991250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:57.157248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.157151 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:57.556300 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.556267 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:57.876656 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.876617 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j"] Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877021 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" containerName="payload-pre-processing" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877034 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" containerName="payload-pre-processing" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877041 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877048 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877061 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877067 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877137 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877146 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab84f2b9-223b-4549-8ec6-941cf9e9b9c3" containerName="payload-pre-processing" Jul 24 14:28:57.877186 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.877152 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a30e88-43ed-4049-9619-f744670c1fdb" containerName="istio-proxy" Jul 24 14:28:57.880136 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.880119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.883019 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.882969 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-94af13-openshift-default-dockercfg-r47hm\"" Jul 24 14:28:57.902207 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.902176 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j"] Jul 24 14:28:57.916353 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.916325 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:57.997193 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997158 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b755fdb4-552a-484a-8384-10a14896f7ef-istio-podinfo\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997193 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997197 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5td4l\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-kube-api-access-5td4l\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997218 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-data\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997293 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b755fdb4-552a-484a-8384-10a14896f7ef-istiod-ca-cert\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997318 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-socket\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997356 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-certs\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-credential-socket\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997491 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-istio-token\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:57.997687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:57.997543 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-envoy\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098213 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-credential-socket\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098213 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098211 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-istio-token\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098240 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-envoy\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b755fdb4-552a-484a-8384-10a14896f7ef-istio-podinfo\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5td4l\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-kube-api-access-5td4l\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-data\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098366 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b755fdb4-552a-484a-8384-10a14896f7ef-istiod-ca-cert\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098390 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-socket\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.098488 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098443 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-certs\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.099512 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.098937 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-credential-socket\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.099512 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.099268 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-certs\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.104006 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.099756 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-data\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.104006 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.099827 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-socket\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.104006 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.101424 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b755fdb4-552a-484a-8384-10a14896f7ef-istiod-ca-cert\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.107088 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.107051 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-envoy\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.108733 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.108703 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b755fdb4-552a-484a-8384-10a14896f7ef-istio-podinfo\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.111485 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.111458 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-istio-token\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.111815 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.111798 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5td4l\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-kube-api-access-5td4l\") pod \"e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.191297 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.191193 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:28:58.353718 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.353692 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j"] Jul 24 14:28:58.355848 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:28:58.355819 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb755fdb4_552a_484a_8384_10a14896f7ef.slice/crio-8fe8838a7d1c907839acea9f3e0d49001e6de3723aa57912bf232a319b5db50e WatchSource:0}: Error finding container 8fe8838a7d1c907839acea9f3e0d49001e6de3723aa57912bf232a319b5db50e: Status 404 returned error can't find the container with id 8fe8838a7d1c907839acea9f3e0d49001e6de3723aa57912bf232a319b5db50e Jul 24 14:28:58.358177 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.358133 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:28:58.358261 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.358224 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:28:58.358308 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.358293 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:28:58.545094 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.545057 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:28:58.601725 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.601684 2555 patch_prober.go:28] interesting pod/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:28:58.601976 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:58.601756 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:28:59.311543 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:59.311503 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" event={"ID":"b755fdb4-552a-484a-8384-10a14896f7ef","Type":"ContainerStarted","Data":"0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8"} Jul 24 14:28:59.311955 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:59.311565 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" event={"ID":"b755fdb4-552a-484a-8384-10a14896f7ef","Type":"ContainerStarted","Data":"8fe8838a7d1c907839acea9f3e0d49001e6de3723aa57912bf232a319b5db50e"} Jul 24 14:28:59.336754 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:28:59.336692 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podStartSLOduration=2.336675813 podStartE2EDuration="2.336675813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:28:59.333428835 +0000 UTC m=+2626.128771565" watchObservedRunningTime="2026-07-24 14:28:59.336675813 +0000 UTC m=+2626.132018542" Jul 24 14:29:00.191867 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:00.191828 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:00.197418 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:00.197391 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:00.316913 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:00.316883 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:00.318117 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:00.318090 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:01.454440 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:01.454399 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:01.555048 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:01.554977 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:01.715496 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:01.715405 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:01.935216 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:29:01.935180 2555 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d28ae38_974b_40a7_8f1c_618449bc1a3a.slice/crio-0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d28ae38_974b_40a7_8f1c_618449bc1a3a.slice/crio-conmon-0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9.scope\": RecentStats: unable to find data in memory cache]" Jul 24 14:29:02.125149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.125117 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:02.165056 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.164976 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:29:02.241097 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241056 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istiod-ca-cert\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241097 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241104 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-credential-socket\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241141 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-socket\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241183 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-data\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241235 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wkk9\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-kube-api-access-2wkk9\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241261 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-podinfo\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241282 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-envoy\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241332 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241331 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-certs\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.241637 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241389 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:29:02.241637 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241569 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:29:02.241637 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241590 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:29:02.241800 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241692 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:29:02.241800 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.241729 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-token\") pod \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\" (UID: \"4d28ae38-974b-40a7-8f1c-618449bc1a3a\") " Jul 24 14:29:02.242049 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.242031 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.242130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.242050 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.242130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.242067 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.242130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.242076 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.242130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.242112 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:29:02.243632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.243609 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:29:02.243947 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.243932 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:29:02.244177 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.244154 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-kube-api-access-2wkk9" (OuterVolumeSpecName: "kube-api-access-2wkk9") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "kube-api-access-2wkk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:29:02.244177 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.244158 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4d28ae38-974b-40a7-8f1c-618449bc1a3a" (UID: "4d28ae38-974b-40a7-8f1c-618449bc1a3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:29:02.328922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.328884 2555 generic.go:356] "Generic (PLEG): container finished" podID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerID="0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9" exitCode=0 Jul 24 14:29:02.329121 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.328930 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" Jul 24 14:29:02.329121 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.328972 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" event={"ID":"4d28ae38-974b-40a7-8f1c-618449bc1a3a","Type":"ContainerDied","Data":"0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9"} Jul 24 14:29:02.329121 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.329047 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8" event={"ID":"4d28ae38-974b-40a7-8f1c-618449bc1a3a","Type":"ContainerDied","Data":"cb02513dd8b9bde95d618c9da6cca6cb0f347db0f61871cbf4a023b00d2fb84e"} Jul 24 14:29:02.329121 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.329071 2555 scope.go:117] "RemoveContainer" containerID="0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9" Jul 24 14:29:02.338680 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.338658 2555 scope.go:117] "RemoveContainer" containerID="baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df" Jul 24 14:29:02.343385 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.343360 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.343491 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.343387 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wkk9\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-kube-api-access-2wkk9\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.343491 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.343398 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.343491 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.343410 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.343491 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.343423 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d28ae38-974b-40a7-8f1c-618449bc1a3a-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:29:02.344916 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.344888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:02.347929 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.347912 2555 scope.go:117] "RemoveContainer" containerID="0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9" Jul 24 14:29:02.348208 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:29:02.348191 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9\": container with ID starting with 0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9 not found: ID does not exist" containerID="0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9" Jul 24 14:29:02.348252 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.348216 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9"} err="failed to get container status \"0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9\": rpc error: code = NotFound desc = could not find container \"0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9\": container with ID starting with 0c05765de1ec138630cab90b169b00be42b26fdfdf01728584f7d2dd41cf3fb9 not found: ID does not exist" Jul 24 14:29:02.348252 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.348233 2555 scope.go:117] "RemoveContainer" containerID="baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df" Jul 24 14:29:02.348482 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:29:02.348463 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df\": container with ID starting with baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df not found: ID does not exist" containerID="baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df" Jul 24 14:29:02.348533 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.348491 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df"} err="failed to get container status \"baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df\": rpc error: code = NotFound desc = could not find container \"baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df\": container with ID starting with baab999c15dc053142d781ed4f65015f570048846fa899a8def5a4fde8b767df not found: ID does not exist" Jul 24 14:29:02.356815 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.356759 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8"] Jul 24 14:29:02.363772 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.363737 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-45b6b7-openshift-default-79c8fd5dcd-hswx8"] Jul 24 14:29:02.458947 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.458918 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:02.650468 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.650363 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r"] Jul 24 14:29:02.650853 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.650835 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" Jul 24 14:29:02.650930 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.650856 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" Jul 24 14:29:02.651019 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.650965 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" Jul 24 14:29:02.655730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.655707 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.659751 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.659722 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-94af13\"" Jul 24 14:29:02.660155 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.660133 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-94af13-dockercfg-hf9f7\"" Jul 24 14:29:02.675599 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.675563 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r"] Jul 24 14:29:02.748287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.748250 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9737589-566c-4f1a-9ee1-9823b35f1830-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.748287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.748288 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2759q\" (UniqueName: \"kubernetes.io/projected/c9737589-566c-4f1a-9ee1-9823b35f1830-kube-api-access-2759q\") pod \"payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.849601 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.849551 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9737589-566c-4f1a-9ee1-9823b35f1830-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.849601 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.849603 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2759q\" (UniqueName: \"kubernetes.io/projected/c9737589-566c-4f1a-9ee1-9823b35f1830-kube-api-access-2759q\") pod \"payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.850318 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.850296 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9737589-566c-4f1a-9ee1-9823b35f1830-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.859150 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.859121 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2759q\" (UniqueName: \"kubernetes.io/projected/c9737589-566c-4f1a-9ee1-9823b35f1830-kube-api-access-2759q\") pod \"payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:02.968140 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:02.968019 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:03.143534 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:03.143504 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r"] Jul 24 14:29:03.145679 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:29:03.145652 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9737589_566c_4f1a_9ee1_9823b35f1830.slice/crio-0072473a0021b34485bf0c9bbc35373b969d1b1041aefed9c38f28188903faf7 WatchSource:0}: Error finding container 0072473a0021b34485bf0c9bbc35373b969d1b1041aefed9c38f28188903faf7: Status 404 returned error can't find the container with id 0072473a0021b34485bf0c9bbc35373b969d1b1041aefed9c38f28188903faf7 Jul 24 14:29:03.337642 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:03.337608 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" event={"ID":"c9737589-566c-4f1a-9ee1-9823b35f1830","Type":"ContainerStarted","Data":"1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21"} Jul 24 14:29:03.337841 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:03.337659 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" event={"ID":"c9737589-566c-4f1a-9ee1-9823b35f1830","Type":"ContainerStarted","Data":"0072473a0021b34485bf0c9bbc35373b969d1b1041aefed9c38f28188903faf7"} Jul 24 14:29:03.337841 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:03.337702 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:03.389034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:03.388945 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" podStartSLOduration=1.388928333 podStartE2EDuration="1.388928333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:29:03.3881355 +0000 UTC m=+2630.183478231" watchObservedRunningTime="2026-07-24 14:29:03.388928333 +0000 UTC m=+2630.184271066" Jul 24 14:29:03.750009 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:03.749952 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" path="/var/lib/kubelet/pods/4d28ae38-974b-40a7-8f1c-618449bc1a3a/volumes" Jul 24 14:29:05.863733 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:05.863702 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:06.249622 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.249518 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:06.604543 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.604504 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc"] Jul 24 14:29:06.605033 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.605014 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" Jul 24 14:29:06.605142 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.605035 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" Jul 24 14:29:06.605142 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.605132 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d28ae38-974b-40a7-8f1c-618449bc1a3a" containerName="istio-proxy" Jul 24 14:29:06.608513 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.608490 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.611533 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.611508 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-938af8-openshift-default-dockercfg-z4fwm\"" Jul 24 14:29:06.632632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.632593 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc"] Jul 24 14:29:06.663657 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.663622 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:06.689658 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689607 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vgjq\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-kube-api-access-7vgjq\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.689851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-certs\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.689851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689760 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-podinfo\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.689851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689797 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-data\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.689851 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689840 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a46fb7d7-58cd-4e97-940f-b45d43777419-istiod-ca-cert\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.690061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689879 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-socket\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.690061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689910 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-credential-socket\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.690061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.689950 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-envoy\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.690061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.690008 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-token\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791433 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791393 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-data\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791454 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a46fb7d7-58cd-4e97-940f-b45d43777419-istiod-ca-cert\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791495 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-socket\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791520 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-credential-socket\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791560 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-envoy\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791597 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-token\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.791662 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791659 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vgjq\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-kube-api-access-7vgjq\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791685 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-certs\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791735 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-podinfo\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791897 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-data\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.791939 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-socket\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792297 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.792266 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-certs\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792395 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.792378 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-credential-socket\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.792918 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.792835 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a46fb7d7-58cd-4e97-940f-b45d43777419-istiod-ca-cert\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.794285 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.794255 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-envoy\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.794833 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.794812 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-podinfo\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.801102 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.801056 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-token\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.801324 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.801304 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vgjq\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-kube-api-access-7vgjq\") pod \"e2e-shared-b-938af8-openshift-default-698486947b-d9vgc\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:06.920213 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:06.920096 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:07.084538 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.084415 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:07.086781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.086745 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc"] Jul 24 14:29:07.087308 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:29:07.087278 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda46fb7d7_58cd_4e97_940f_b45d43777419.slice/crio-9e0d559968df6ecb470f6d03e10276359944d77f5971c80ba6a35202253eda3e WatchSource:0}: Error finding container 9e0d559968df6ecb470f6d03e10276359944d77f5971c80ba6a35202253eda3e: Status 404 returned error can't find the container with id 9e0d559968df6ecb470f6d03e10276359944d77f5971c80ba6a35202253eda3e Jul 24 14:29:07.094224 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.094181 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:07.094303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.094275 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:07.094350 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.094316 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:07.360097 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.360061 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" event={"ID":"a46fb7d7-58cd-4e97-940f-b45d43777419","Type":"ContainerStarted","Data":"ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6"} Jul 24 14:29:07.360309 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.360124 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" event={"ID":"a46fb7d7-58cd-4e97-940f-b45d43777419","Type":"ContainerStarted","Data":"9e0d559968df6ecb470f6d03e10276359944d77f5971c80ba6a35202253eda3e"} Jul 24 14:29:07.381727 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.381664 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" podStartSLOduration=1.381641096 podStartE2EDuration="1.381641096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:29:07.381348952 +0000 UTC m=+2634.176691683" watchObservedRunningTime="2026-07-24 14:29:07.381641096 +0000 UTC m=+2634.176983827" Jul 24 14:29:07.920212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:07.920172 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:08.926227 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:08.926196 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:09.370062 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:09.370027 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:09.371208 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:09.371178 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:09.452429 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:09.452398 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:09.567150 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:09.567115 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:09.662451 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:09.662349 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:10.198008 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:10.195509 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:10.321255 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:10.321219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:13.888682 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:13.888641 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:14.344338 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:14.344309 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:29:15.636451 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:15.636146 2555 patch_prober.go:28] interesting pod/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": read tcp 10.133.0.2:46538->10.133.0.74:15021: read: connection reset by peer" start-of-body= Jul 24 14:29:15.636451 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:15.636229 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": read tcp 10.133.0.2:46538->10.133.0.74:15021: read: connection reset by peer" Jul 24 14:29:16.419664 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:16.419629 2555 generic.go:356] "Generic (PLEG): container finished" podID="b755fdb4-552a-484a-8384-10a14896f7ef" containerID="0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8" exitCode=0 Jul 24 14:29:16.419847 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:16.419705 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" event={"ID":"b755fdb4-552a-484a-8384-10a14896f7ef","Type":"ContainerDied","Data":"0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8"} Jul 24 14:29:16.420189 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:16.420173 2555 scope.go:117] "RemoveContainer" containerID="0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8" Jul 24 14:29:16.421246 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:16.421216 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:16.421314 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:16.421297 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:16.421350 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:16.421336 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:17.432145 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:17.432100 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" event={"ID":"b755fdb4-552a-484a-8384-10a14896f7ef","Type":"ContainerStarted","Data":"d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404"} Jul 24 14:29:18.192477 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:18.192432 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:18.192812 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:18.192572 2555 patch_prober.go:28] interesting pod/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 14:29:18.192812 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:18.192615 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 14:29:19.195831 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:19.192409 2555 patch_prober.go:28] interesting pod/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 14:29:19.195831 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:19.192493 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 14:29:20.192031 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:20.191964 2555 patch_prober.go:28] interesting pod/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 14:29:20.192275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:20.192081 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 14:29:21.192069 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:21.192021 2555 patch_prober.go:28] interesting pod/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 14:29:21.192577 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:21.192093 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 14:29:22.196505 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:22.196468 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:22.197031 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:22.196864 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:22.197970 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:22.197941 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:29:23.468416 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:23.468385 2555 generic.go:356] "Generic (PLEG): container finished" podID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerID="ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6" exitCode=0 Jul 24 14:29:23.468858 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:23.468463 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" event={"ID":"a46fb7d7-58cd-4e97-940f-b45d43777419","Type":"ContainerDied","Data":"ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6"} Jul 24 14:29:23.469043 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:23.469027 2555 scope.go:117] "RemoveContainer" containerID="ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6" Jul 24 14:29:23.470117 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:23.470085 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:23.470212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:23.470158 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:23.470212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:23.470197 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:29:24.370351 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:24.370313 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:24.476485 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:24.476452 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" event={"ID":"a46fb7d7-58cd-4e97-940f-b45d43777419","Type":"ContainerStarted","Data":"d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea"} Jul 24 14:29:24.920557 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:24.920511 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:24.921903 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:24.921874 2555 patch_prober.go:28] interesting pod/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 14:29:24.922064 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:24.921925 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 14:29:25.921458 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:25.921421 2555 patch_prober.go:28] interesting pod/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 14:29:25.921872 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:25.921478 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 14:29:26.920526 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:26.920489 2555 patch_prober.go:28] interesting pod/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 14:29:26.920710 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:26.920564 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 14:29:27.921237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:27.921193 2555 patch_prober.go:28] interesting pod/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 14:29:27.921661 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:27.921271 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 14:29:28.925434 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:28.925403 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:28.925887 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:28.925668 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:28.926620 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:28.926600 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:29:33.015370 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:33.015318 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:38.385353 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:38.385320 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:38.508181 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:38.508144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:29:52.884051 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:29:52.884015 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:30:09.179464 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:30:09.179377 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:30:41.663367 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:30:41.663328 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:32:06.923864 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:32:06.923827 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:32:08.390157 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:32:08.390123 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:32:19.779308 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:32:19.779269 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:32:44.874004 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:32:44.873949 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:32:59.192886 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:32:59.192850 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:33:33.922731 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:33:33.922700 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:33:34.721637 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:33:34.721601 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:33:48.455545 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:33:48.455512 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:34:25.181546 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:34:25.181510 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:06.315289 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:06.315250 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:07.196085 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:07.196050 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:26.831573 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:26.831542 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:27.122619 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.122530 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc"] Jul 24 14:35:27.126099 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.126074 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.129096 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.129071 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-106097-openshift-default-dockercfg-z68rr\"" Jul 24 14:35:27.138344 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.138316 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc"] Jul 24 14:35:27.157736 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.157706 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:27.228149 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228108 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11415c0-e9df-415e-b924-d31163c74db3-istiod-ca-cert\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228399 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228173 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-istio-token\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228399 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228259 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-data\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228399 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228309 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-socket\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228399 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-credential-socket\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228399 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228377 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-envoy\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228638 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-certs\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228638 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228442 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgqch\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-kube-api-access-jgqch\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.228638 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.228476 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11415c0-e9df-415e-b924-d31163c74db3-istio-podinfo\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.329826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.329782 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-credential-socket\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.329826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.329823 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-envoy\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330074 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.329970 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-certs\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330074 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330025 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgqch\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-kube-api-access-jgqch\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330074 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11415c0-e9df-415e-b924-d31163c74db3-istio-podinfo\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330210 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11415c0-e9df-415e-b924-d31163c74db3-istiod-ca-cert\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330210 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-istio-token\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330307 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330272 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-data\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330307 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330293 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-credential-socket\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330422 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330313 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-socket\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330422 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330340 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-certs\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330637 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330613 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-socket\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.330738 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.330690 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-data\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.331311 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.331290 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11415c0-e9df-415e-b924-d31163c74db3-istiod-ca-cert\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.332496 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.332473 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-envoy\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.332702 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.332684 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11415c0-e9df-415e-b924-d31163c74db3-istio-podinfo\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.339136 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.339109 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-istio-token\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.339285 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.339271 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgqch\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-kube-api-access-jgqch\") pod \"e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.437272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.437177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:27.598136 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.598078 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc"] Jul 24 14:35:27.600381 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:35:27.600354 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11415c0_e9df_415e_b924_d31163c74db3.slice/crio-33d88c1389218b47916327f77d206d8394cc7ff5401025eec6840430fe31f456 WatchSource:0}: Error finding container 33d88c1389218b47916327f77d206d8394cc7ff5401025eec6840430fe31f456: Status 404 returned error can't find the container with id 33d88c1389218b47916327f77d206d8394cc7ff5401025eec6840430fe31f456 Jul 24 14:35:27.603165 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.603122 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:27.603264 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.603225 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:27.603324 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.603272 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:27.854246 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:27.854213 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:28.350776 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:28.350739 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" event={"ID":"d11415c0-e9df-415e-b924-d31163c74db3","Type":"ContainerStarted","Data":"08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84"} Jul 24 14:35:28.350971 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:28.350802 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" event={"ID":"d11415c0-e9df-415e-b924-d31163c74db3","Type":"ContainerStarted","Data":"33d88c1389218b47916327f77d206d8394cc7ff5401025eec6840430fe31f456"} Jul 24 14:35:28.377452 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:28.377392 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" podStartSLOduration=1.377375631 podStartE2EDuration="1.377375631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:35:28.375335161 +0000 UTC m=+3015.170677891" watchObservedRunningTime="2026-07-24 14:35:28.377375631 +0000 UTC m=+3015.172718361" Jul 24 14:35:28.437740 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:28.437694 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:28.442916 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:28.442881 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:29.356233 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:29.356206 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:29.357309 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:29.357291 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:30.333084 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:30.333045 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:30.436217 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:30.436186 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:30.531636 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:30.531598 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:30.636409 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:30.636306 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:31.180230 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:31.180197 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:31.270594 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:31.270564 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:34.868336 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:34.868294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:48.461857 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:48.461818 2555 generic.go:356] "Generic (PLEG): container finished" podID="d11415c0-e9df-415e-b924-d31163c74db3" containerID="08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84" exitCode=0 Jul 24 14:35:48.462370 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:48.461894 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" event={"ID":"d11415c0-e9df-415e-b924-d31163c74db3","Type":"ContainerDied","Data":"08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84"} Jul 24 14:35:48.462370 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:48.462337 2555 scope.go:117] "RemoveContainer" containerID="08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84" Jul 24 14:35:48.463429 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:48.463399 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:48.463524 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:48.463467 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:48.463524 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:48.463501 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:49.469523 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:49.469488 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" event={"ID":"d11415c0-e9df-415e-b924-d31163c74db3","Type":"ContainerStarted","Data":"03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f"} Jul 24 14:35:50.438101 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:50.438061 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:50.438299 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:50.438275 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 24 14:35:50.438348 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:50.438324 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 24 14:35:51.437810 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:51.437767 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 24 14:35:51.438261 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:51.437841 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 24 14:35:52.438278 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:52.438240 2555 patch_prober.go:28] interesting pod/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Jul 24 14:35:52.438688 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:52.438301 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Jul 24 14:35:53.452024 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:53.451970 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:53.452442 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:53.452290 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:53.453061 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:53.453044 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:35:56.093789 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.093751 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:56.393904 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.393806 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc"] Jul 24 14:35:56.397443 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.397413 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.400795 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.400762 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-fde43a-openshift-default-dockercfg-5kz2h\"" Jul 24 14:35:56.411184 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.411157 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc"] Jul 24 14:35:56.432450 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.432423 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:56.449880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.449843 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-data\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450076 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.449892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-socket\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450076 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.449914 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-certs\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450076 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.449950 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-podinfo\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450076 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.450026 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-credential-socket\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450076 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.450074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istiod-ca-cert\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.450147 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-envoy\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.450188 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phgz5\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-kube-api-access-phgz5\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.450248 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.450219 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-token\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.550722 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.550686 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istiod-ca-cert\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.550906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.550737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-envoy\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.550906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.550856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phgz5\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-kube-api-access-phgz5\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551037 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.550927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-token\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551037 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551029 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-data\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551139 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551064 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-socket\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551139 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551097 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-certs\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551139 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551128 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-podinfo\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551282 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-credential-socket\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551489 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551462 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-data\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551567 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-credential-socket\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551611 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-certs\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551795 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551768 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istiod-ca-cert\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.551884 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.551843 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-socket\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.553303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.553272 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-envoy\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.553968 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.553947 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-podinfo\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.560614 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.560586 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-token\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.560921 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.560901 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phgz5\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-kube-api-access-phgz5\") pod \"e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.697735 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.697652 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:56.710666 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.710632 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:56.869391 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.869304 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc"] Jul 24 14:35:56.872099 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:35:56.872065 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b3a1f10_50ec_411e_8e1d_cf1f407535e2.slice/crio-af431a4ec2edec6e51b428575a2f6c9f09494cba55d0c88da522c6e85485d253 WatchSource:0}: Error finding container af431a4ec2edec6e51b428575a2f6c9f09494cba55d0c88da522c6e85485d253: Status 404 returned error can't find the container with id af431a4ec2edec6e51b428575a2f6c9f09494cba55d0c88da522c6e85485d253 Jul 24 14:35:56.874472 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.874440 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:56.874552 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.874505 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:56.874552 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:56.874533 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:35:57.520218 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:57.520188 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" event={"ID":"6b3a1f10-50ec-411e-8e1d-cf1f407535e2","Type":"ContainerStarted","Data":"b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f"} Jul 24 14:35:57.520570 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:57.520243 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" event={"ID":"6b3a1f10-50ec-411e-8e1d-cf1f407535e2","Type":"ContainerStarted","Data":"af431a4ec2edec6e51b428575a2f6c9f09494cba55d0c88da522c6e85485d253"} Jul 24 14:35:57.541615 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:57.541564 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podStartSLOduration=1.541547292 podStartE2EDuration="1.541547292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:35:57.540752856 +0000 UTC m=+3044.336095586" watchObservedRunningTime="2026-07-24 14:35:57.541547292 +0000 UTC m=+3044.336890022" Jul 24 14:35:57.711474 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:57.711435 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:57.717000 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:57.716958 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:58.526316 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:58.526283 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:58.527303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:58.527285 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:35:59.504745 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:59.504716 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:59.608019 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:59.607950 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:59.705325 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:59.705294 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:35:59.848053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:35:59.848014 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:36:00.171255 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:00.171168 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:36:00.273058 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:00.273024 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:36:04.070183 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:04.070105 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:36:17.637404 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:17.637371 2555 generic.go:356] "Generic (PLEG): container finished" podID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerID="b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f" exitCode=0 Jul 24 14:36:17.637845 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:17.637448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" event={"ID":"6b3a1f10-50ec-411e-8e1d-cf1f407535e2","Type":"ContainerDied","Data":"b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f"} Jul 24 14:36:17.637926 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:17.637909 2555 scope.go:117] "RemoveContainer" containerID="b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f" Jul 24 14:36:17.639011 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:17.638957 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:36:17.639119 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:17.639055 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:36:17.639119 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:17.639090 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:36:18.646312 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:18.646276 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" event={"ID":"6b3a1f10-50ec-411e-8e1d-cf1f407535e2","Type":"ContainerStarted","Data":"edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6"} Jul 24 14:36:18.711720 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:18.711673 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:36:18.712024 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:18.711975 2555 patch_prober.go:28] interesting pod/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Jul 24 14:36:18.712139 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:18.712060 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Jul 24 14:36:19.711481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:19.711441 2555 patch_prober.go:28] interesting pod/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Jul 24 14:36:19.712053 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:19.711505 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Jul 24 14:36:20.711366 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:20.711313 2555 patch_prober.go:28] interesting pod/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Jul 24 14:36:20.711564 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:20.711388 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Jul 24 14:36:21.431074 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:21.431022 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nftmt_02b5d03f-8032-42aa-ae94-c3521ea07a7d/payload-processing/0.log" Jul 24 14:36:21.712098 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:21.712004 2555 patch_prober.go:28] interesting pod/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Jul 24 14:36:21.712098 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:21.712065 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Jul 24 14:36:22.711589 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:22.711549 2555 patch_prober.go:28] interesting pod/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body= Jul 24 14:36:22.711788 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:22.711610 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" Jul 24 14:36:23.715193 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:23.715157 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:36:23.715585 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:23.715546 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:36:23.716201 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:23.716186 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:36:28.391975 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:28.391938 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:36:58.304039 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:36:58.303974 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:00.426441 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:00.426407 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nftmt_02b5d03f-8032-42aa-ae94-c3521ea07a7d/payload-processing/0.log" Jul 24 14:37:00.852547 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:00.852508 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:01.156301 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.156210 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh"] Jul 24 14:37:01.159553 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.159532 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.163074 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.163042 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-57f095-openshift-default-dockercfg-5px7g\"" Jul 24 14:37:01.173220 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.173185 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh"] Jul 24 14:37:01.187700 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.187650 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:01.274581 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.274545 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:01.333313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333038 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-envoy\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333091 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-podinfo\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-credential-socket\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333211 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-token\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333273 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-socket\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333306 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmj8g\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-kube-api-access-mmj8g\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333814 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333340 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-data\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333814 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333370 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.333814 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.333429 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-certs\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.434629 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434521 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-token\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.434629 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-socket\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.434629 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434621 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmj8g\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-kube-api-access-mmj8g\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-data\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434675 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434722 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-certs\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-envoy\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434804 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-podinfo\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.434868 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-credential-socket\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.435108 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-socket\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.435125 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-data\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435272 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.435251 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-certs\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435640 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.435287 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-credential-socket\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.435640 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.435506 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istiod-ca-cert\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.437475 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.437455 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-envoy\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.437597 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.437578 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-podinfo\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.443750 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.443726 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-token\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.443928 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.443906 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmj8g\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-kube-api-access-mmj8g\") pod \"e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.471698 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.471660 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:01.627528 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.627466 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh"] Jul 24 14:37:01.630324 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:37:01.630297 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7459e8a_2ba4_4d42_95b2_4f3f9a6903b9.slice/crio-206dcf41891b4b9f7f2535970c53133b6606435b4fc89e7b4a0ac7341e33223a WatchSource:0}: Error finding container 206dcf41891b4b9f7f2535970c53133b6606435b4fc89e7b4a0ac7341e33223a: Status 404 returned error can't find the container with id 206dcf41891b4b9f7f2535970c53133b6606435b4fc89e7b4a0ac7341e33223a Jul 24 14:37:01.633078 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.633044 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:37:01.633160 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.633116 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:37:01.633160 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.633147 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:37:01.890806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.890427 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" event={"ID":"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9","Type":"ContainerStarted","Data":"ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2"} Jul 24 14:37:01.890806 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.890527 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" event={"ID":"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9","Type":"ContainerStarted","Data":"206dcf41891b4b9f7f2535970c53133b6606435b4fc89e7b4a0ac7341e33223a"} Jul 24 14:37:01.913610 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:01.913560 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podStartSLOduration=0.913538512 podStartE2EDuration="913.538512ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:37:01.91203219 +0000 UTC m=+3108.707374933" watchObservedRunningTime="2026-07-24 14:37:01.913538512 +0000 UTC m=+3108.708881242" Jul 24 14:37:02.471855 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:02.471816 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:02.477343 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:02.477316 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:02.896163 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:02.896123 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:02.897362 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:02.897338 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:03.876524 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:03.876493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:03.976912 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:03.976877 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:04.062552 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:04.062505 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:04.165340 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:04.165244 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:04.771208 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:04.771174 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:04.869164 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:04.869125 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:08.385362 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:08.385323 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:17.897854 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.897759 2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Jul 24 14:37:17.897854 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.897828 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Jul 24 14:37:17.986665 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.986632 2555 generic.go:356] "Generic (PLEG): container finished" podID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerID="ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2" exitCode=0 Jul 24 14:37:17.986873 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.986712 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" event={"ID":"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9","Type":"ContainerDied","Data":"ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2"} Jul 24 14:37:17.987257 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.987238 2555 scope.go:117] "RemoveContainer" containerID="ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2" Jul 24 14:37:17.988484 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.988453 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:37:17.988561 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.988524 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:37:17.988561 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:17.988556 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 24 14:37:18.999630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:18.998806 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" event={"ID":"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9","Type":"ContainerStarted","Data":"d09d2b0f852fa57bb933bbcf7f7f256295e6ca8e28d49c89fb18dca510b47f48"} Jul 24 14:37:19.472763 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:19.472725 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:19.473036 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:19.473012 2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Jul 24 14:37:19.473094 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:19.473064 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Jul 24 14:37:20.058237 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:20.058200 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:20.472605 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:20.472398 2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Jul 24 14:37:20.472605 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:20.472471 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Jul 24 14:37:21.472487 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:21.472444 2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Jul 24 14:37:21.472942 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:21.472510 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Jul 24 14:37:22.472251 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:22.472212 2555 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body= Jul 24 14:37:22.472436 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:22.472270 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" Jul 24 14:37:23.476459 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:23.476427 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:23.476842 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:23.476811 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:23.477546 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:23.477525 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:29.913778 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:29.913736 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:29.989179 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:29.989144 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:30.086155 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:30.086114 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:34.427303 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:34.427193 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:34.812409 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:34.812369 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:40.660823 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:40.660781 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:40.736416 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:40.736384 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:40.917037 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:40.916939 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh"] Jul 24 14:37:40.917217 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:40.917194 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" containerID="cri-o://d09d2b0f852fa57bb933bbcf7f7f256295e6ca8e28d49c89fb18dca510b47f48" gracePeriod=30 Jul 24 14:37:40.945957 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:40.945915 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:41.188703 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:41.188604 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:37:46.150653 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.150612 2555 generic.go:356] "Generic (PLEG): container finished" podID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerID="d09d2b0f852fa57bb933bbcf7f7f256295e6ca8e28d49c89fb18dca510b47f48" exitCode=0 Jul 24 14:37:46.151146 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.150664 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" event={"ID":"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9","Type":"ContainerDied","Data":"d09d2b0f852fa57bb933bbcf7f7f256295e6ca8e28d49c89fb18dca510b47f48"} Jul 24 14:37:46.151146 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.150747 2555 scope.go:117] "RemoveContainer" containerID="ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2" Jul 24 14:37:46.176616 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:37:46.176592 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2\": container with ID starting with ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2 not found: ID does not exist" containerID="ea6dc4868fd91724976b476843883b2f3ec3db00739457a5bb42c4fa1c2652c2" Jul 24 14:37:46.178391 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.178374 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:46.200396 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200267 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-podinfo\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200450 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-socket\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200515 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istiod-ca-cert\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200568 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-credential-socket\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200606 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-certs\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200631 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-envoy\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200722 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmj8g\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-kube-api-access-mmj8g\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200750 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-data\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.200924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.200796 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-token\") pod \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\" (UID: \"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9\") " Jul 24 14:37:46.201124 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.201097 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:37:46.203539 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.201227 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.203539 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.202540 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:37:46.203539 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.202614 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:37:46.203539 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.202679 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:37:46.203539 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.203322 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:37:46.203906 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.203812 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:37:46.204593 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.204552 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:37:46.204767 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.204623 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:37:46.205096 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.205068 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-kube-api-access-mmj8g" (OuterVolumeSpecName: "kube-api-access-mmj8g") pod "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" (UID: "b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9"). InnerVolumeSpecName "kube-api-access-mmj8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:37:46.302215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302180 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302210 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302215 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302220 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302464 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302229 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302464 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302238 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302464 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302247 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmj8g\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-kube-api-access-mmj8g\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302464 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302255 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:46.302464 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:46.302263 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:37:47.157635 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:47.157603 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" Jul 24 14:37:47.157635 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:47.157628 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh" event={"ID":"b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9","Type":"ContainerDied","Data":"206dcf41891b4b9f7f2535970c53133b6606435b4fc89e7b4a0ac7341e33223a"} Jul 24 14:37:47.158178 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:47.157690 2555 scope.go:117] "RemoveContainer" containerID="d09d2b0f852fa57bb933bbcf7f7f256295e6ca8e28d49c89fb18dca510b47f48" Jul 24 14:37:47.185013 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:47.184956 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh"] Jul 24 14:37:47.191527 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:47.191492 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-57f095-openshift-default-6fcc8568c6-nq5dh"] Jul 24 14:37:47.750028 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:37:47.749974 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" path="/var/lib/kubelet/pods/b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9/volumes" Jul 24 14:38:00.222362 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:00.222328 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:00.296879 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:00.296849 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:00.363627 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:00.363595 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:04.804912 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:04.804875 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:05.208484 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:05.208391 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:10.280712 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:10.280674 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:30.267854 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:30.267820 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:45.918299 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:45.918265 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:46.017255 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:46.017219 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:46.347896 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:46.347856 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:47.474141 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:47.474102 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:53.114919 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:53.114872 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:58.923161 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:58.923121 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:59.027610 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:59.027573 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:59.249419 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:59.249329 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc"] Jul 24 14:38:59.249588 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:59.249566 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" containerID="cri-o://03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f" gracePeriod=30 Jul 24 14:38:59.289928 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:59.289888 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:38:59.488083 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:38:59.488041 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:04.506349 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.506318 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:39:04.576423 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.576387 2555 generic.go:356] "Generic (PLEG): container finished" podID="d11415c0-e9df-415e-b924-d31163c74db3" containerID="03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f" exitCode=0 Jul 24 14:39:04.576602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.576438 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" Jul 24 14:39:04.576602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.576474 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" event={"ID":"d11415c0-e9df-415e-b924-d31163c74db3","Type":"ContainerDied","Data":"03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f"} Jul 24 14:39:04.576602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.576532 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc" event={"ID":"d11415c0-e9df-415e-b924-d31163c74db3","Type":"ContainerDied","Data":"33d88c1389218b47916327f77d206d8394cc7ff5401025eec6840430fe31f456"} Jul 24 14:39:04.576602 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.576557 2555 scope.go:117] "RemoveContainer" containerID="03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f" Jul 24 14:39:04.585941 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.585920 2555 scope.go:117] "RemoveContainer" containerID="08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84" Jul 24 14:39:04.595064 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.595040 2555 scope.go:117] "RemoveContainer" containerID="03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f" Jul 24 14:39:04.595425 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:04.595401 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f\": container with ID starting with 03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f not found: ID does not exist" containerID="03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f" Jul 24 14:39:04.595520 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.595440 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f"} err="failed to get container status \"03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f\": rpc error: code = NotFound desc = could not find container \"03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f\": container with ID starting with 03d18bd5216c7ca3e7a01a8c9c71b8b27cf5185cd5ca4c1e4b8d1714a484ac9f not found: ID does not exist" Jul 24 14:39:04.595520 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.595467 2555 scope.go:117] "RemoveContainer" containerID="08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84" Jul 24 14:39:04.595762 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:04.595742 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84\": container with ID starting with 08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84 not found: ID does not exist" containerID="08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84" Jul 24 14:39:04.595804 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.595772 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84"} err="failed to get container status \"08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84\": rpc error: code = NotFound desc = could not find container \"08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84\": container with ID starting with 08219d21bdd5049f17616b4f05181aaa30989f6d21b528d936109a2f0b068e84 not found: ID does not exist" Jul 24 14:39:04.604171 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604144 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11415c0-e9df-415e-b924-d31163c74db3-istio-podinfo\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604184 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgqch\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-kube-api-access-jgqch\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604210 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-credential-socket\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604231 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-envoy\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604287 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604271 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-istio-token\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604483 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604310 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-socket\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604483 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604344 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-data\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604483 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604414 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11415c0-e9df-415e-b924-d31163c74db3-istiod-ca-cert\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604483 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604437 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-certs\") pod \"d11415c0-e9df-415e-b924-d31163c74db3\" (UID: \"d11415c0-e9df-415e-b924-d31163c74db3\") " Jul 24 14:39:04.604749 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604482 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:04.604802 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604764 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.604802 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604785 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d11415c0-e9df-415e-b924-d31163c74db3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:39:04.604904 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.604854 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:04.605157 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.605014 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:04.605821 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.605794 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-data" (OuterVolumeSpecName: "istio-data") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:04.606582 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.606560 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d11415c0-e9df-415e-b924-d31163c74db3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:39:04.606729 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.606709 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:04.606807 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.606771 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-kube-api-access-jgqch" (OuterVolumeSpecName: "kube-api-access-jgqch") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "kube-api-access-jgqch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:04.606854 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.606806 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-istio-token" (OuterVolumeSpecName: "istio-token") pod "d11415c0-e9df-415e-b924-d31163c74db3" (UID: "d11415c0-e9df-415e-b924-d31163c74db3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:04.705732 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705690 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11415c0-e9df-415e-b924-d31163c74db3-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.705732 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705725 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgqch\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-kube-api-access-jgqch\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.705732 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705738 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.706059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705750 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11415c0-e9df-415e-b924-d31163c74db3-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.706059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705761 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.706059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705772 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.706059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705783 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11415c0-e9df-415e-b924-d31163c74db3-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.706059 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.705794 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11415c0-e9df-415e-b924-d31163c74db3-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:04.902498 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.902456 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc"] Jul 24 14:39:04.910511 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:04.910467 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-106097-openshift-default-f6f586c4d-lbttc"] Jul 24 14:39:05.750434 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:05.750391 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d11415c0-e9df-415e-b924-d31163c74db3" path="/var/lib/kubelet/pods/d11415c0-e9df-415e-b924-d31163c74db3/volumes" Jul 24 14:39:11.058623 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:11.058585 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:11.158671 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:11.158615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:15.830594 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:15.830533 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:15.884416 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:15.884377 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:20.805439 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:20.805395 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:20.905047 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:20.905012 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:21.090376 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:21.090287 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc"] Jul 24 14:39:21.090578 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:21.090553 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" containerID="cri-o://edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6" gracePeriod=30 Jul 24 14:39:21.113060 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:21.113020 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:21.360704 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:21.360615 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:23.717929 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:23.717888 2555 patch_prober.go:28] interesting pod/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 14:39:23.718344 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:23.717938 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 14:39:26.357854 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.357783 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:39:26.530518 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530477 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-credential-socket\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.530730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530560 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-token\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.530730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530607 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istiod-ca-cert\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.530730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530645 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-socket\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.530730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530691 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-data\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.530730 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530716 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-envoy\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.531034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530776 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phgz5\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-kube-api-access-phgz5\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.531034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530808 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-certs\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.531034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530846 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-podinfo\") pod \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\" (UID: \"6b3a1f10-50ec-411e-8e1d-cf1f407535e2\") " Jul 24 14:39:26.531034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.530847 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:26.531034 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531021 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:26.531315 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531172 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:39:26.531315 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531265 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:26.531315 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531306 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.531460 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531323 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.531460 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531338 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.531570 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.531546 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:26.533632 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.533603 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:26.533755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.533644 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-kube-api-access-phgz5" (OuterVolumeSpecName: "kube-api-access-phgz5") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "kube-api-access-phgz5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:26.533755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.533694 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:26.533755 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.533704 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6b3a1f10-50ec-411e-8e1d-cf1f407535e2" (UID: "6b3a1f10-50ec-411e-8e1d-cf1f407535e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:39:26.632785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.632687 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phgz5\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-kube-api-access-phgz5\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.632785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.632724 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.632785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.632733 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.632785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.632741 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.632785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.632749 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.632785 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.632756 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b3a1f10-50ec-411e-8e1d-cf1f407535e2-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:26.692529 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.692499 2555 generic.go:356] "Generic (PLEG): container finished" podID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerID="edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6" exitCode=0 Jul 24 14:39:26.692709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.692548 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" Jul 24 14:39:26.692709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.692585 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" event={"ID":"6b3a1f10-50ec-411e-8e1d-cf1f407535e2","Type":"ContainerDied","Data":"edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6"} Jul 24 14:39:26.692709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.692639 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc" event={"ID":"6b3a1f10-50ec-411e-8e1d-cf1f407535e2","Type":"ContainerDied","Data":"af431a4ec2edec6e51b428575a2f6c9f09494cba55d0c88da522c6e85485d253"} Jul 24 14:39:26.692709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.692663 2555 scope.go:117] "RemoveContainer" containerID="edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6" Jul 24 14:39:26.704175 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.704150 2555 scope.go:117] "RemoveContainer" containerID="b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f" Jul 24 14:39:26.714539 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.714517 2555 scope.go:117] "RemoveContainer" containerID="edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6" Jul 24 14:39:26.714876 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:26.714851 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6\": container with ID starting with edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6 not found: ID does not exist" containerID="edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6" Jul 24 14:39:26.714922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.714891 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6"} err="failed to get container status \"edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6\": rpc error: code = NotFound desc = could not find container \"edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6\": container with ID starting with edd61997714cba3025ea04a10a234c854cc3366c6743f631f4b81e42b8acf4d6 not found: ID does not exist" Jul 24 14:39:26.714922 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.714919 2555 scope.go:117] "RemoveContainer" containerID="b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f" Jul 24 14:39:26.715295 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:26.715252 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f\": container with ID starting with b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f not found: ID does not exist" containerID="b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f" Jul 24 14:39:26.715295 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.715279 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f"} err="failed to get container status \"b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f\": rpc error: code = NotFound desc = could not find container \"b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f\": container with ID starting with b2ac3bde024cbffa6f775a23f92406e8202914ffacbf3a251bf226b34f09e36f not found: ID does not exist" Jul 24 14:39:26.716733 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.716709 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc"] Jul 24 14:39:26.720203 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:26.720178 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-fde43a-openshift-default-589cd77d78-2tkxc"] Jul 24 14:39:27.749997 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:27.749944 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" path="/var/lib/kubelet/pods/6b3a1f10-50ec-411e-8e1d-cf1f407535e2/volumes" Jul 24 14:39:31.956293 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:31.956260 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:31.979811 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:31.979773 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r"] Jul 24 14:39:31.980127 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:31.980100 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" podUID="c9737589-566c-4f1a-9ee1-9823b35f1830" containerName="payload-processing" containerID="cri-o://1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21" gracePeriod=30 Jul 24 14:39:32.026895 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.026852 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:32.127792 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.127749 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:32.254891 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.254867 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:39:32.392744 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.392704 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9737589-566c-4f1a-9ee1-9823b35f1830-plugins-config-volume\") pod \"c9737589-566c-4f1a-9ee1-9823b35f1830\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " Jul 24 14:39:32.392935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.392783 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2759q\" (UniqueName: \"kubernetes.io/projected/c9737589-566c-4f1a-9ee1-9823b35f1830-kube-api-access-2759q\") pod \"c9737589-566c-4f1a-9ee1-9823b35f1830\" (UID: \"c9737589-566c-4f1a-9ee1-9823b35f1830\") " Jul 24 14:39:32.393226 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.393202 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9737589-566c-4f1a-9ee1-9823b35f1830-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9737589-566c-4f1a-9ee1-9823b35f1830" (UID: "c9737589-566c-4f1a-9ee1-9823b35f1830"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:39:32.395182 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.395155 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9737589-566c-4f1a-9ee1-9823b35f1830-kube-api-access-2759q" (OuterVolumeSpecName: "kube-api-access-2759q") pod "c9737589-566c-4f1a-9ee1-9823b35f1830" (UID: "c9737589-566c-4f1a-9ee1-9823b35f1830"). InnerVolumeSpecName "kube-api-access-2759q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:32.494567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.494463 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2759q\" (UniqueName: \"kubernetes.io/projected/c9737589-566c-4f1a-9ee1-9823b35f1830-kube-api-access-2759q\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:32.494567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.494498 2555 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9737589-566c-4f1a-9ee1-9823b35f1830-plugins-config-volume\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:32.726226 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.726191 2555 generic.go:356] "Generic (PLEG): container finished" podID="c9737589-566c-4f1a-9ee1-9823b35f1830" containerID="1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21" exitCode=0 Jul 24 14:39:32.726410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.726237 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" Jul 24 14:39:32.726410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.726283 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" event={"ID":"c9737589-566c-4f1a-9ee1-9823b35f1830","Type":"ContainerDied","Data":"1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21"} Jul 24 14:39:32.726410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.726334 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r" event={"ID":"c9737589-566c-4f1a-9ee1-9823b35f1830","Type":"ContainerDied","Data":"0072473a0021b34485bf0c9bbc35373b969d1b1041aefed9c38f28188903faf7"} Jul 24 14:39:32.726410 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.726362 2555 scope.go:117] "RemoveContainer" containerID="1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21" Jul 24 14:39:32.735932 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.735910 2555 scope.go:117] "RemoveContainer" containerID="1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21" Jul 24 14:39:32.736300 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:32.736276 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21\": container with ID starting with 1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21 not found: ID does not exist" containerID="1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21" Jul 24 14:39:32.736366 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.736310 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21"} err="failed to get container status \"1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21\": rpc error: code = NotFound desc = could not find container \"1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21\": container with ID starting with 1eb39a87cf97c6a1fee5285a9564eb32a1907ab49c42edcdb93f115837405f21 not found: ID does not exist" Jul 24 14:39:32.749008 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.748895 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r"] Jul 24 14:39:32.752585 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:32.752551 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-94af13-5846ffbcdb-n9w9r"] Jul 24 14:39:33.749618 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:33.749572 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9737589-566c-4f1a-9ee1-9823b35f1830" path="/var/lib/kubelet/pods/c9737589-566c-4f1a-9ee1-9823b35f1830/volumes" Jul 24 14:39:36.579534 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:36.579493 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:36.968772 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:36.968677 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:43.037623 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:43.037587 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:43.110693 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:43.110649 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:43.296935 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:43.296900 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j"] Jul 24 14:39:43.297261 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:43.297234 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" containerID="cri-o://d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404" gracePeriod=30 Jul 24 14:39:43.314895 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:43.314854 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:43.553871 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:43.553780 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:48.564092 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.564018 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:39:48.650457 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650415 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-envoy\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650457 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650463 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b755fdb4-552a-484a-8384-10a14896f7ef-istiod-ca-cert\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650517 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-certs\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650566 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b755fdb4-552a-484a-8384-10a14896f7ef-istio-podinfo\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650598 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-istio-token\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650635 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5td4l\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-kube-api-access-5td4l\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650677 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650659 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-credential-socket\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650729 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-data\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650756 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-socket\") pod \"b755fdb4-552a-484a-8384-10a14896f7ef\" (UID: \"b755fdb4-552a-484a-8384-10a14896f7ef\") " Jul 24 14:39:48.650927 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650831 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:48.651129 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.650937 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b755fdb4-552a-484a-8384-10a14896f7ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:39:48.651190 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.651129 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:48.651190 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.651114 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:48.651341 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.651315 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.651419 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.651341 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.651419 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.651357 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b755fdb4-552a-484a-8384-10a14896f7ef-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.651419 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.651376 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.652212 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.652184 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:48.652892 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.652873 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:39:48.653014 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.652976 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b755fdb4-552a-484a-8384-10a14896f7ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:39:48.653231 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.653211 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:48.653305 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.653287 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-kube-api-access-5td4l" (OuterVolumeSpecName: "kube-api-access-5td4l") pod "b755fdb4-552a-484a-8384-10a14896f7ef" (UID: "b755fdb4-552a-484a-8384-10a14896f7ef"). InnerVolumeSpecName "kube-api-access-5td4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:39:48.752158 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.752119 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.752158 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.752152 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b755fdb4-552a-484a-8384-10a14896f7ef-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.752158 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.752160 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.752158 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.752170 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5td4l\" (UniqueName: \"kubernetes.io/projected/b755fdb4-552a-484a-8384-10a14896f7ef-kube-api-access-5td4l\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.752434 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.752179 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b755fdb4-552a-484a-8384-10a14896f7ef-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:39:48.816549 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.816462 2555 generic.go:356] "Generic (PLEG): container finished" podID="b755fdb4-552a-484a-8384-10a14896f7ef" containerID="d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404" exitCode=0 Jul 24 14:39:48.816549 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.816511 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" event={"ID":"b755fdb4-552a-484a-8384-10a14896f7ef","Type":"ContainerDied","Data":"d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404"} Jul 24 14:39:48.816549 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.816523 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" Jul 24 14:39:48.816774 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.816577 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j" event={"ID":"b755fdb4-552a-484a-8384-10a14896f7ef","Type":"ContainerDied","Data":"8fe8838a7d1c907839acea9f3e0d49001e6de3723aa57912bf232a319b5db50e"} Jul 24 14:39:48.816774 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.816600 2555 scope.go:117] "RemoveContainer" containerID="d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404" Jul 24 14:39:48.826335 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.826315 2555 scope.go:117] "RemoveContainer" containerID="0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8" Jul 24 14:39:48.835694 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.835578 2555 scope.go:117] "RemoveContainer" containerID="d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404" Jul 24 14:39:48.837809 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:48.837775 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404\": container with ID starting with d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404 not found: ID does not exist" containerID="d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404" Jul 24 14:39:48.837958 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.837818 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404"} err="failed to get container status \"d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404\": rpc error: code = NotFound desc = could not find container \"d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404\": container with ID starting with d77445f67c1fef58f71348e564ce83aa607853b09dd58d3aff28910ab987a404 not found: ID does not exist" Jul 24 14:39:48.837958 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.837845 2555 scope.go:117] "RemoveContainer" containerID="0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8" Jul 24 14:39:48.838403 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:39:48.838378 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8\": container with ID starting with 0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8 not found: ID does not exist" containerID="0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8" Jul 24 14:39:48.838519 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.838407 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8"} err="failed to get container status \"0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8\": rpc error: code = NotFound desc = could not find container \"0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8\": container with ID starting with 0d669d81ec6f328a2c1257031239b7adee492455cc6a976ffec7a684df1b47d8 not found: ID does not exist" Jul 24 14:39:48.839288 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.839264 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j"] Jul 24 14:39:48.842386 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:48.842362 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-94af13-openshift-default-6f55dd6dd8-ktw4j"] Jul 24 14:39:49.750042 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:49.750013 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" path="/var/lib/kubelet/pods/b755fdb4-552a-484a-8384-10a14896f7ef/volumes" Jul 24 14:39:54.290152 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:54.290118 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:54.365033 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:54.364967 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:54.436358 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:54.436329 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:39:58.758609 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:39:58.758571 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:00.747205 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:00.747164 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:05.133107 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:05.133070 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:05.207641 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:05.207584 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:05.409169 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:05.409074 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc"] Jul 24 14:40:05.409340 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:05.409310 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" containerID="cri-o://d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea" gracePeriod=30 Jul 24 14:40:05.437703 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:05.437668 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:05.688756 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:05.688658 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:06.107457 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:06.107423 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fb7b944c7-vjs5r_ed8af88c-6c72-4b23-8594-19e694e41072/authorino/0.log" Jul 24 14:40:10.674830 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.674761 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:40:10.765953 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.765922 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-token\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766183 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766005 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-envoy\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766183 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766062 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vgjq\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-kube-api-access-7vgjq\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766183 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766090 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-podinfo\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766365 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766233 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-certs\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766365 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766272 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-credential-socket\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766365 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766330 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a46fb7d7-58cd-4e97-940f-b45d43777419-istiod-ca-cert\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766365 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766358 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-data\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766405 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-socket\") pod \"a46fb7d7-58cd-4e97-940f-b45d43777419\" (UID: \"a46fb7d7-58cd-4e97-940f-b45d43777419\") " Jul 24 14:40:10.766567 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766550 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:40:10.766817 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766753 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:40:10.766817 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766801 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-credential-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.766817 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.766764 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a46fb7d7-58cd-4e97-940f-b45d43777419-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:40:10.767601 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.767564 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:40:10.767830 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.767805 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-data" (OuterVolumeSpecName: "istio-data") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:40:10.768826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.768799 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-token" (OuterVolumeSpecName: "istio-token") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:40:10.768826 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.768819 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-kube-api-access-7vgjq" (OuterVolumeSpecName: "kube-api-access-7vgjq") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "kube-api-access-7vgjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:40:10.769015 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.768899 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:40:10.769075 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.769049 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a46fb7d7-58cd-4e97-940f-b45d43777419" (UID: "a46fb7d7-58cd-4e97-940f-b45d43777419"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:40:10.868275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868219 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-certs\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868264 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a46fb7d7-58cd-4e97-940f-b45d43777419-istiod-ca-cert\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868276 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-data\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868284 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-workload-socket\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868275 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868291 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-token\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868599 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868301 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-envoy\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868599 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868309 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vgjq\" (UniqueName: \"kubernetes.io/projected/a46fb7d7-58cd-4e97-940f-b45d43777419-kube-api-access-7vgjq\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.868599 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.868318 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a46fb7d7-58cd-4e97-940f-b45d43777419-istio-podinfo\") on node \"ip-10-0-143-204.ec2.internal\" DevicePath \"\"" Jul 24 14:40:10.930243 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.930149 2555 generic.go:356] "Generic (PLEG): container finished" podID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerID="d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea" exitCode=0 Jul 24 14:40:10.930243 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.930206 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" Jul 24 14:40:10.930429 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.930211 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" event={"ID":"a46fb7d7-58cd-4e97-940f-b45d43777419","Type":"ContainerDied","Data":"d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea"} Jul 24 14:40:10.930429 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.930328 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc" event={"ID":"a46fb7d7-58cd-4e97-940f-b45d43777419","Type":"ContainerDied","Data":"9e0d559968df6ecb470f6d03e10276359944d77f5971c80ba6a35202253eda3e"} Jul 24 14:40:10.930429 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.930351 2555 scope.go:117] "RemoveContainer" containerID="d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea" Jul 24 14:40:10.940315 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.940294 2555 scope.go:117] "RemoveContainer" containerID="ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6" Jul 24 14:40:10.949477 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.949453 2555 scope.go:117] "RemoveContainer" containerID="d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea" Jul 24 14:40:10.949797 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:40:10.949774 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea\": container with ID starting with d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea not found: ID does not exist" containerID="d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea" Jul 24 14:40:10.949872 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.949808 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea"} err="failed to get container status \"d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea\": rpc error: code = NotFound desc = could not find container \"d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea\": container with ID starting with d6164f0a8c25bd7188c1ab3d844df3340f34bf5ff55b3ca4b06ba8388dadb1ea not found: ID does not exist" Jul 24 14:40:10.949872 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.949828 2555 scope.go:117] "RemoveContainer" containerID="ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6" Jul 24 14:40:10.950111 ip-10-0-143-204 kubenswrapper[2555]: E0724 14:40:10.950091 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6\": container with ID starting with ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6 not found: ID does not exist" containerID="ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6" Jul 24 14:40:10.950176 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.950122 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6"} err="failed to get container status \"ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6\": rpc error: code = NotFound desc = could not find container \"ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6\": container with ID starting with ace4e2d0593533a8a3ca93dfd8e947dc6642448803f896b5326c6d31c3e595d6 not found: ID does not exist" Jul 24 14:40:10.954627 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.954596 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc"] Jul 24 14:40:10.958536 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:10.958507 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-938af8-openshift-default-698486947b-d9vgc"] Jul 24 14:40:11.749276 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:11.749242 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" path="/var/lib/kubelet/pods/a46fb7d7-58cd-4e97-940f-b45d43777419/volumes" Jul 24 14:40:12.111557 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:12.111526 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nkmxv_bc0b970c-0087-47b7-906e-bdb81fb53851/server/0.log" Jul 24 14:40:12.231128 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:12.231100 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b9kt9_61c5caff-6c6b-4082-9da9-0198004dc5c2/manager/0.log" Jul 24 14:40:12.610446 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:12.610425 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd96dfd5f-slsbr_b3b06285-f2fb-4be5-b911-20c6ac3bde46/copy-manifests/0.log" Jul 24 14:40:12.617728 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:12.617702 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd96dfd5f-slsbr_b3b06285-f2fb-4be5-b911-20c6ac3bde46/manager/0.log" Jul 24 14:40:14.838354 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:14.838320 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fb7b944c7-vjs5r_ed8af88c-6c72-4b23-8594-19e694e41072/authorino/0.log" Jul 24 14:40:15.082255 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:15.082223 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qt5zm_f646fb79-0037-4ab6-84c2-fba0fa9c23fa/manager/0.log" Jul 24 14:40:15.206049 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:15.205932 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jqccp_261390f9-e358-4e1e-9ad7-91258b821347/kuadrant-console-plugin/0.log" Jul 24 14:40:15.327554 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:15.327518 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-sztsl_51ef3fae-305b-4ed6-a06d-54471d352f74/registry-server/0.log" Jul 24 14:40:15.451701 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:15.451669 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8r25n_37bcc377-9cb5-4e7a-858b-febfa7ff7681/manager/0.log" Jul 24 14:40:16.048454 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:16.048424 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9_941b394c-91cb-4af3-90cb-8ad0751070ab/istio-proxy/0.log" Jul 24 14:40:16.520306 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:16.520217 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/1.log" Jul 24 14:40:16.647079 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:16.647006 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-rsm9z_ac51fa65-d5d0-40ae-8e74-53802b5bc606/payload-pre-processing/0.log" Jul 24 14:40:16.769388 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:16.769353 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nftmt_02b5d03f-8032-42aa-ae94-c3521ea07a7d/payload-processing/0.log" Jul 24 14:40:17.250630 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:17.250603 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-f89kz_9d39bf1f-d096-45b8-b980-02b367cd0857/main/0.log" Jul 24 14:40:17.493081 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:17.493053 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-zh54d_d9a88726-271a-4df2-9b87-4431f7462a0d/main/0.log" Jul 24 14:40:23.134130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134084 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mmxsf/must-gather-7xgfq"] Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134462 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134475 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134485 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134491 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134499 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134504 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134512 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134517 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134523 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134529 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134536 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9737589-566c-4f1a-9ee1-9823b35f1830" containerName="payload-processing" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134541 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9737589-566c-4f1a-9ee1-9823b35f1830" containerName="payload-processing" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134558 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134563 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134570 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134575 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" Jul 24 14:40:23.134575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134583 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134588 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134595 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134601 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134609 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134614 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134669 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134677 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134682 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134688 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134693 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b755fdb4-552a-484a-8384-10a14896f7ef" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134698 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9737589-566c-4f1a-9ee1-9823b35f1830" containerName="payload-processing" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134704 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a46fb7d7-58cd-4e97-940f-b45d43777419" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134710 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134716 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d11415c0-e9df-415e-b924-d31163c74db3" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134827 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b3a1f10-50ec-411e-8e1d-cf1f407535e2" containerName="istio-proxy" Jul 24 14:40:23.135123 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.134833 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7459e8a-2ba4-4d42-95b2-4f3f9a6903b9" containerName="istio-proxy" Jul 24 14:40:23.138130 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.138107 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.140889 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.140866 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mmxsf\"/\"kube-root-ca.crt\"" Jul 24 14:40:23.141316 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.141297 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mmxsf\"/\"default-dockercfg-jjn97\"" Jul 24 14:40:23.141610 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.141594 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mmxsf\"/\"openshift-service-ca.crt\"" Jul 24 14:40:23.149829 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.149801 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmxsf/must-gather-7xgfq"] Jul 24 14:40:23.162999 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.162948 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed-must-gather-output\") pod \"must-gather-7xgfq\" (UID: \"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed\") " pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.163196 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.163150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzl5d\" (UniqueName: \"kubernetes.io/projected/2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed-kube-api-access-tzl5d\") pod \"must-gather-7xgfq\" (UID: \"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed\") " pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.163859 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.163815 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:23.260188 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.260127 2555 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" limit=1024 Jul 24 14:40:23.264230 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.264202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzl5d\" (UniqueName: \"kubernetes.io/projected/2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed-kube-api-access-tzl5d\") pod \"must-gather-7xgfq\" (UID: \"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed\") " pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.264372 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.264297 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed-must-gather-output\") pod \"must-gather-7xgfq\" (UID: \"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed\") " pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.264647 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.264629 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed-must-gather-output\") pod \"must-gather-7xgfq\" (UID: \"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed\") " pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.273278 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.273254 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzl5d\" (UniqueName: \"kubernetes.io/projected/2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed-kube-api-access-tzl5d\") pod \"must-gather-7xgfq\" (UID: \"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed\") " pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.447920 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.447826 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmxsf/must-gather-7xgfq" Jul 24 14:40:23.584709 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.584680 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmxsf/must-gather-7xgfq"] Jul 24 14:40:23.587190 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:40:23.587159 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bd3c1b4_5e90_45f0_8e20_a9eeaf1cd5ed.slice/crio-32b128302be608066992c3e8d2d50c442a6bbf0ea56394713c118d5b9a123b4b WatchSource:0}: Error finding container 32b128302be608066992c3e8d2d50c442a6bbf0ea56394713c118d5b9a123b4b: Status 404 returned error can't find the container with id 32b128302be608066992c3e8d2d50c442a6bbf0ea56394713c118d5b9a123b4b Jul 24 14:40:23.589182 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.589161 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 14:40:23.996861 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:23.996823 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmxsf/must-gather-7xgfq" event={"ID":"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed","Type":"ContainerStarted","Data":"32b128302be608066992c3e8d2d50c442a6bbf0ea56394713c118d5b9a123b4b"} Jul 24 14:40:25.006771 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:25.006725 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmxsf/must-gather-7xgfq" event={"ID":"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed","Type":"ContainerStarted","Data":"0bac4500df6fd9c850e0548a310c4204d7715a63916cf2c0b356e698bd11c9a6"} Jul 24 14:40:25.007416 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:25.006789 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmxsf/must-gather-7xgfq" event={"ID":"2bd3c1b4-5e90-45f0-8e20-a9eeaf1cd5ed","Type":"ContainerStarted","Data":"0193749e8b4baca28225914216ed31746697751d012118c12882e9f3235c8765"} Jul 24 14:40:25.028296 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:25.028230 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mmxsf/must-gather-7xgfq" podStartSLOduration=1.167962032 podStartE2EDuration="2.028204483s" totalImagesPullingTime="860.242451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:40:25.024913714 +0000 UTC m=+3311.820256443" watchObservedRunningTime="2026-07-24 14:40:25.028204483 +0000 UTC m=+3311.823547213" Jul 24 14:40:26.105321 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:26.105291 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wsr7q_2ce8fd08-5295-4821-9d6f-c7bed4fc260f/global-pull-secret-syncer/0.log" Jul 24 14:40:26.163997 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:26.163952 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9228l_f085458a-3b25-43ca-85a4-d79e0ae4a2f8/konnectivity-agent/0.log" Jul 24 14:40:26.303017 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:26.302968 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-204.ec2.internal_b2b0af06f1bfda6fafb07193c56cc418/haproxy/0.log" Jul 24 14:40:30.942811 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:30.942767 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6fb7b944c7-vjs5r_ed8af88c-6c72-4b23-8594-19e694e41072/authorino/0.log" Jul 24 14:40:31.026260 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:31.026226 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qt5zm_f646fb79-0037-4ab6-84c2-fba0fa9c23fa/manager/0.log" Jul 24 14:40:31.061421 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:31.061379 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-jqccp_261390f9-e358-4e1e-9ad7-91258b821347/kuadrant-console-plugin/0.log" Jul 24 14:40:31.110492 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:31.110456 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-sztsl_51ef3fae-305b-4ed6-a06d-54471d352f74/registry-server/0.log" Jul 24 14:40:31.289501 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:31.289394 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8r25n_37bcc377-9cb5-4e7a-858b-febfa7ff7681/manager/0.log" Jul 24 14:40:33.257297 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:33.257194 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7788c867cc-6sqck_c8564a70-37c5-45a1-bb1f-98f092faef66/metrics-server/0.log" Jul 24 14:40:33.404781 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:33.404749 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-85q2q_06071628-5820-4d75-9ff3-c1519047d07a/node-exporter/0.log" Jul 24 14:40:33.429904 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:33.429880 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-85q2q_06071628-5820-4d75-9ff3-c1519047d07a/kube-rbac-proxy/0.log" Jul 24 14:40:33.455187 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:33.455160 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-85q2q_06071628-5820-4d75-9ff3-c1519047d07a/init-textfile/0.log" Jul 24 14:40:36.666137 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.666093 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j"] Jul 24 14:40:36.674544 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.674508 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.680965 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.680935 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j"] Jul 24 14:40:36.730249 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.730211 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-lib-modules\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.730687 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.730659 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65fgq\" (UniqueName: \"kubernetes.io/projected/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-kube-api-access-65fgq\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.730871 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.730857 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-sys\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.731043 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.731027 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-proc\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.731172 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.731158 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-podres\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832269 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-proc\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832296 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-podres\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832375 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-proc\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832427 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-lib-modules\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832446 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-podres\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832481 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832463 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65fgq\" (UniqueName: \"kubernetes.io/projected/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-kube-api-access-65fgq\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832582 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-sys\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832592 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-lib-modules\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.832743 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.832659 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-sys\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.842518 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.842476 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65fgq\" (UniqueName: \"kubernetes.io/projected/ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e-kube-api-access-65fgq\") pod \"perf-node-gather-daemonset-4sb8j\" (UID: \"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e\") " pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:36.899253 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.899216 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58bc85ccc6-7rx6k_4d68954f-8c36-4438-8ff2-4d73f5274f4b/console/0.log" Jul 24 14:40:36.990320 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:36.990214 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:37.175575 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:37.175437 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j"] Jul 24 14:40:37.178102 ip-10-0-143-204 kubenswrapper[2555]: W0724 14:40:37.178066 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poded2cd9e7_c046_4ac0_80ca_6b0182b7b25e.slice/crio-6eaf7aa8fbb07a0fda9f7166bd8174bc22b8ab54153c9ce2232ce534cc05f491 WatchSource:0}: Error finding container 6eaf7aa8fbb07a0fda9f7166bd8174bc22b8ab54153c9ce2232ce534cc05f491: Status 404 returned error can't find the container with id 6eaf7aa8fbb07a0fda9f7166bd8174bc22b8ab54153c9ce2232ce534cc05f491 Jul 24 14:40:38.110640 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.110583 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" event={"ID":"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e","Type":"ContainerStarted","Data":"35d0eeda78300bcfb97a3b2c4409cafb413be619eccec75819d8bd3845e97df7"} Jul 24 14:40:38.111172 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.110654 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" event={"ID":"ed2cd9e7-c046-4ac0-80ca-6b0182b7b25e","Type":"ContainerStarted","Data":"6eaf7aa8fbb07a0fda9f7166bd8174bc22b8ab54153c9ce2232ce534cc05f491"} Jul 24 14:40:38.111172 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.110946 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:38.130769 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.130700 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" podStartSLOduration=2.130677089 podStartE2EDuration="2.130677089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:40:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:40:38.128999289 +0000 UTC m=+3324.924342020" watchObservedRunningTime="2026-07-24 14:40:38.130677089 +0000 UTC m=+3324.926019821" Jul 24 14:40:38.695029 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.694999 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sc55j_4949ccfd-f2b4-44b4-9e00-2228bb2ad367/dns/0.log" Jul 24 14:40:38.718257 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.718209 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sc55j_4949ccfd-f2b4-44b4-9e00-2228bb2ad367/kube-rbac-proxy/0.log" Jul 24 14:40:38.741708 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:38.741680 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cvnzc_01d4d8d2-be23-425c-b318-811a3cd71ab8/dns-node-resolver/0.log" Jul 24 14:40:39.358951 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:39.358913 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6564b854b7-j9x9w_9ff6530b-8347-4bbf-a3ec-934e0a1b898d/registry/0.log" Jul 24 14:40:39.443807 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:39.443780 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n9vmm_be143ea6-d2ee-4709-8fd8-de79390abd1f/node-ca/0.log" Jul 24 14:40:40.401086 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:40.401057 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9_941b394c-91cb-4af3-90cb-8ad0751070ab/istio-proxy/0.log" Jul 24 14:40:40.655090 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:40.655009 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/0.log" Jul 24 14:40:40.689354 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:40.689318 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/1.log" Jul 24 14:40:40.727064 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:40.727025 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-rsm9z_ac51fa65-d5d0-40ae-8e74-53802b5bc606/payload-pre-processing/0.log" Jul 24 14:40:40.766006 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:40.765956 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nftmt_02b5d03f-8032-42aa-ae94-c3521ea07a7d/payload-processing/0.log" Jul 24 14:40:41.694284 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:41.694243 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xknnz_1c1a5e07-f00e-4480-b0f1-f2fa6624461a/serve-healthcheck-canary/0.log" Jul 24 14:40:42.212274 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:42.212240 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqc9b_62578128-8b67-4150-aeb4-12efb712bfe1/kube-rbac-proxy/0.log" Jul 24 14:40:42.248098 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:42.248072 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqc9b_62578128-8b67-4150-aeb4-12efb712bfe1/exporter/0.log" Jul 24 14:40:42.282649 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:42.282623 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqc9b_62578128-8b67-4150-aeb4-12efb712bfe1/extractor/0.log" Jul 24 14:40:44.129202 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:44.129171 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mmxsf/perf-node-gather-daemonset-4sb8j" Jul 24 14:40:45.023431 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:45.023336 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nkmxv_bc0b970c-0087-47b7-906e-bdb81fb53851/server/0.log" Jul 24 14:40:45.123786 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:45.123732 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b9kt9_61c5caff-6c6b-4082-9da9-0198004dc5c2/manager/0.log" Jul 24 14:40:45.312748 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:45.312717 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd96dfd5f-slsbr_b3b06285-f2fb-4be5-b911-20c6ac3bde46/manager/0.log" Jul 24 14:40:45.338301 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:45.338269 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6cd96dfd5f-slsbr_b3b06285-f2fb-4be5-b911-20c6ac3bde46/copy-manifests/0.log" Jul 24 14:40:46.783938 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:46.783835 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84df8c9cc-h9njz_082170f1-beec-444c-ba1b-3d67b930a43b/manager/0.log" Jul 24 14:40:47.268954 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:47.268925 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9_941b394c-91cb-4af3-90cb-8ad0751070ab/istio-proxy/0.log" Jul 24 14:40:47.504363 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:47.504328 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/0.log" Jul 24 14:40:47.533890 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:47.533799 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/1.log" Jul 24 14:40:47.574164 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:47.574128 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-rsm9z_ac51fa65-d5d0-40ae-8e74-53802b5bc606/payload-pre-processing/0.log" Jul 24 14:40:47.618367 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:47.618336 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nftmt_02b5d03f-8032-42aa-ae94-c3521ea07a7d/payload-processing/0.log" Jul 24 14:40:49.159107 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:49.159073 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wvhr9_941b394c-91cb-4af3-90cb-8ad0751070ab/istio-proxy/0.log" Jul 24 14:40:49.432463 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:49.432380 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/0.log" Jul 24 14:40:49.467620 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:49.467582 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-qkcrn_dd838d0a-45bf-4e91-8473-41bb74497fc1/istio-proxy/1.log" Jul 24 14:40:49.496495 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:49.496463 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-rsm9z_ac51fa65-d5d0-40ae-8e74-53802b5bc606/payload-pre-processing/0.log" Jul 24 14:40:49.520924 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:49.520891 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-nftmt_02b5d03f-8032-42aa-ae94-c3521ea07a7d/payload-processing/0.log" Jul 24 14:40:52.012178 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:52.012144 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-qzflz_c8c19755-f162-4a5b-a0f4-d503f7a270c5/migrator/0.log" Jul 24 14:40:52.037323 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:52.037286 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-qzflz_c8c19755-f162-4a5b-a0f4-d503f7a270c5/graceful-termination/0.log" Jul 24 14:40:53.604485 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.604448 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/kube-multus-additional-cni-plugins/0.log" Jul 24 14:40:53.625973 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.625940 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/egress-router-binary-copy/0.log" Jul 24 14:40:53.646880 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.646850 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/cni-plugins/0.log" Jul 24 14:40:53.668276 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.668247 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/bond-cni-plugin/0.log" Jul 24 14:40:53.689027 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.688997 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/routeoverride-cni/0.log" Jul 24 14:40:53.711837 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.711806 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/whereabouts-cni-bincopy/0.log" Jul 24 14:40:53.733313 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:53.733286 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kb9dl_f7a62685-2897-4199-bc3f-8f4471dbebd6/whereabouts-cni/0.log" Jul 24 14:40:54.015401 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:54.015369 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xpkrh_421b243e-5770-4c8b-985b-6f58e9e8899e/kube-multus/0.log" Jul 24 14:40:54.148227 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:54.148199 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ql85p_3c5e9d55-41ee-4f12-ae25-74947f9bc6f5/network-metrics-daemon/0.log" Jul 24 14:40:54.169103 ip-10-0-143-204 kubenswrapper[2555]: I0724 14:40:54.169078 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ql85p_3c5e9d55-41ee-4f12-ae25-74947f9bc6f5/kube-rbac-proxy/0.log"