Jul 29 12:25:26.985738 ip-10-0-137-159 systemd[1]: Starting Kubernetes Kubelet... Jul 29 12:25:27.379715 ip-10-0-137-159 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 12:25:27.379715 ip-10-0-137-159 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 12:25:27.379715 ip-10-0-137-159 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 12:25:27.379715 ip-10-0-137-159 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 12:25:27.379715 ip-10-0-137-159 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 12:25:27.380283 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.379780 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 12:25:27.383973 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383958 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 12:25:27.383973 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383972 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 12:25:27.383973 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383975 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383978 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383981 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383985 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383988 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383991 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383994 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.383997 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384000 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384002 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384005 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384008 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384011 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384013 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384016 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384018 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384021 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384024 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384026 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384030 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 12:25:27.384072 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384039 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384043 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384046 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384049 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384053 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384055 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384058 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384061 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384064 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384067 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384069 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384072 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384074 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384077 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384080 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384082 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384085 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384088 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384090 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 12:25:27.384667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384093 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384096 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384098 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384101 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384103 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384105 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384108 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384110 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384113 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384115 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384117 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384120 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384122 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384124 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384127 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384130 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384135 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384138 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384140 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 12:25:27.385109 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384143 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384145 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384148 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384151 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384153 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384155 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384158 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384161 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384163 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384165 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384169 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384172 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384174 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384177 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384179 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384182 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384184 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384186 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384189 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384191 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 12:25:27.385589 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384194 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384196 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384199 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384201 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384204 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384206 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384208 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384211 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384214 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384217 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384220 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384222 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384225 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384228 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384230 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384232 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384235 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384237 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384239 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384897 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 12:25:27.386070 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384904 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384908 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384912 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384915 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384919 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384922 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384924 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384927 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384930 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384933 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384935 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384937 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384941 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384943 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384946 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384949 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384952 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384954 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384957 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 12:25:27.386552 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384959 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384961 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384964 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384966 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384969 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384971 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384974 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384976 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384979 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384982 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384984 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384987 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384989 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384991 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384994 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384996 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.384998 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385001 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385003 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385006 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 12:25:27.387009 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385008 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385011 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385013 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385015 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385019 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385021 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385024 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385027 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385030 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385034 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385036 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385039 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385041 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385044 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385046 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385049 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385051 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385054 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385057 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 12:25:27.387495 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385059 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385062 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385064 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385067 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385069 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385072 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385074 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385076 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385079 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385081 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385085 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385087 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385090 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385093 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385095 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385097 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385100 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385104 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385106 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385108 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 12:25:27.387944 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385111 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385114 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385116 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385119 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385121 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385124 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385126 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385129 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385131 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385133 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385136 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385139 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385143 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385146 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385148 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385151 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385153 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385156 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385158 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385160 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 12:25:27.388426 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385524 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385529 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385532 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385534 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385538 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385540 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385542 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385545 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385547 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385551 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385554 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385557 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385559 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385562 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385564 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385567 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385569 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385572 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385574 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385577 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 12:25:27.388930 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385579 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385583 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385587 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385590 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385593 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385596 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385599 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385602 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385604 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385606 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385609 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385612 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385614 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385616 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385619 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385621 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385623 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385628 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385631 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 12:25:27.389405 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385634 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385636 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385641 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385644 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385647 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385649 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385652 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385654 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385657 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385659 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385662 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385664 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385667 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385669 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385671 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385674 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385676 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385678 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385681 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 12:25:27.389880 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385683 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385686 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385689 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385691 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385693 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385695 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385698 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385700 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385702 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385705 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385707 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385710 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385712 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385715 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385717 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385720 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385723 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385726 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385728 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385734 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 12:25:27.390357 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385736 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385738 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385741 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385743 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385746 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385748 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385751 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385755 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385758 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385761 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385764 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385767 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385769 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385771 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385774 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385776 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385779 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385781 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385784 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385786 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 12:25:27.390894 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.385789 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385851 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385857 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385863 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385867 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385871 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385874 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385880 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385883 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385887 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385890 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385894 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385897 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385900 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385903 2563 flags.go:64] FLAG: --cgroup-root="" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385906 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385909 2563 flags.go:64] FLAG: --client-ca-file="" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385912 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385915 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385918 2563 flags.go:64] FLAG: --cluster-domain="" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385921 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385924 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385927 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385930 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 12:25:27.391405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385933 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385936 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385939 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385942 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385945 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385948 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385951 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385953 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385956 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385960 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385963 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385965 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385968 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385971 2563 flags.go:64] FLAG: --enable-server="true" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385974 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385979 2563 flags.go:64] FLAG: --event-burst="100" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385982 2563 flags.go:64] FLAG: --event-qps="50" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385985 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385988 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385992 2563 flags.go:64] FLAG: --eviction-hard="" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385996 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.385999 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386001 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386004 2563 flags.go:64] FLAG: --eviction-soft="" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386007 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 12:25:27.392240 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386009 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386012 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386015 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386017 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386020 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386023 2563 flags.go:64] FLAG: --feature-gates="" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386026 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386029 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386032 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386035 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386038 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386040 2563 flags.go:64] FLAG: --help="false" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386043 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386046 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386049 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386051 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386055 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386058 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386061 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386063 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386066 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386069 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386074 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386077 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 12:25:27.392914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386080 2563 flags.go:64] FLAG: --kube-reserved="" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386083 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386086 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386089 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386092 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386095 2563 flags.go:64] FLAG: --lock-file="" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386098 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386100 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386103 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386108 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386110 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386113 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386116 2563 flags.go:64] FLAG: --logging-format="text" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386118 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386121 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386124 2563 flags.go:64] FLAG: --manifest-url="" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386126 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386130 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386133 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386136 2563 flags.go:64] FLAG: --max-pods="110" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386139 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386142 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386145 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386147 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386150 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 12:25:27.393522 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386153 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386156 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386162 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386165 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386168 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386171 2563 flags.go:64] FLAG: --pod-cidr="" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386176 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386181 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386183 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386186 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386190 2563 flags.go:64] FLAG: --port="10250" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386193 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386196 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-065c517af2ea68af6" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386199 2563 flags.go:64] FLAG: --qos-reserved="" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386202 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386205 2563 flags.go:64] FLAG: --register-node="true" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386207 2563 flags.go:64] FLAG: --register-with-taints="" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386215 2563 flags.go:64] FLAG: --registry-burst="10" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386217 2563 flags.go:64] FLAG: --registry-qps="5" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386220 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386222 2563 flags.go:64] FLAG: --reserved-memory="" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386226 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386229 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386232 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386235 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 12:25:27.394126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386237 2563 flags.go:64] FLAG: --runonce="false" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386240 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386243 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386246 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386248 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386251 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386254 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386257 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386260 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386262 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386265 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386267 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386270 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386274 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386277 2563 flags.go:64] FLAG: --system-cgroups="" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386280 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386284 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386288 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386291 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386294 2563 flags.go:64] FLAG: --tls-min-version="" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386297 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386300 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386302 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386305 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386308 2563 flags.go:64] FLAG: --v="2" Jul 29 12:25:27.394752 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386311 2563 flags.go:64] FLAG: --version="false" Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386315 2563 flags.go:64] FLAG: --vmodule="" Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386318 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386321 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386425 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386429 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386431 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386434 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386436 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386439 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386441 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386444 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386446 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386448 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386451 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386453 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386468 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386471 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386474 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386476 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 12:25:27.395372 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386479 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386482 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386494 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386497 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386501 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386503 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386506 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386509 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386511 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386514 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386516 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386520 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386523 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386526 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386529 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386532 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386534 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386537 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386540 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 12:25:27.395862 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386543 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386545 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386548 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386550 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386553 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386556 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386558 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386561 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386564 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386567 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386569 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386572 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386574 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386577 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386579 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386581 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386587 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386591 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386593 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386596 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 12:25:27.396339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386598 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386602 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386605 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386608 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386610 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386612 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386615 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386617 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386619 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386622 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386625 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386627 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386630 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386633 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386637 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386640 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386642 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386645 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386647 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386650 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 12:25:27.397024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386652 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386654 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386657 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386660 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386663 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386666 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386668 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386671 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386673 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386677 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386680 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386682 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386685 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386687 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386690 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386692 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386695 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386697 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386699 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 12:25:27.397578 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386702 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386704 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386707 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386709 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.386711 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.386717 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.393354 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.393370 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.393382 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.393388 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393446 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393451 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393454 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393470 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 12:25:27.398064 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393475 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393480 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393483 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393486 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393489 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393492 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393494 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393497 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393500 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393503 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393505 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393508 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393511 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393513 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393516 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393518 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393521 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393523 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393526 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393529 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 12:25:27.398392 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393532 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393534 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393537 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393540 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393543 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393552 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393555 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393558 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393561 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393563 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393565 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393568 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393571 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393573 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393576 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393578 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393580 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393583 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393585 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393588 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 12:25:27.398878 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393590 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393592 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393595 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393597 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393600 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393602 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393604 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393607 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393609 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393612 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393615 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393617 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393619 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393622 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393624 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393627 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393631 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393636 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393645 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 12:25:27.399343 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393647 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393650 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393652 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393655 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393657 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393660 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393662 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393665 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393668 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393670 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393672 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393675 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393678 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393681 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393684 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393686 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393689 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393691 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393694 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393696 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 12:25:27.399837 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393699 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393701 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393704 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393707 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393711 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393714 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393718 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393721 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393725 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393729 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393733 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393737 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393748 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393752 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393757 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 12:25:27.400300 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393761 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.393770 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393920 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393925 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393928 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393931 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393934 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393937 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393939 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393942 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393944 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393947 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393949 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393952 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393954 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 12:25:27.400685 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393957 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393959 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393963 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393966 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393969 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393971 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393974 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393977 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393979 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393982 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393984 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393987 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393989 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393992 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.393994 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394002 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394004 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394007 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394009 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 12:25:27.401045 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394012 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394014 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394017 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394019 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394021 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394024 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394026 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394029 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394031 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394033 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394036 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394038 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394040 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394043 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394045 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394047 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394049 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394052 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394054 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394057 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 12:25:27.401563 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394059 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394061 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394063 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394066 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394068 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394071 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394074 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394077 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394079 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394087 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394090 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394092 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394094 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394097 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394099 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394102 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394104 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394107 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394109 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394111 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 12:25:27.402024 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394114 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394116 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394118 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394121 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394123 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394126 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394128 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394130 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394133 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394136 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394138 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394141 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394144 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394146 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394149 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394151 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394154 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394156 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394159 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394161 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 12:25:27.402505 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394163 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394166 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394173 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394176 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394179 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394183 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:27.394187 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.394192 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.394305 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.398881 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.400586 2563 server.go:1019] "Starting client certificate rotation" Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.400680 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 12:25:27.402963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.400715 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 12:25:27.403249 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.402963 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 12:25:27.404177 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.404164 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 12:25:27.416089 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.416074 2563 log.go:25] "Validated CRI v1 runtime API" Jul 29 12:25:27.417959 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.417946 2563 log.go:25] "Validated CRI v1 image API" Jul 29 12:25:27.418756 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.418740 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 12:25:27.423177 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.423160 2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d8b185f2-7e59-40c5-b3da-0c120d736758:/dev/nvme0n1p3 de56666a-b9d2-4fa6-a374-9154d78e1ed1:/dev/nvme0n1p4] Jul 29 12:25:27.423237 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.423178 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 12:25:27.428013 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.427918 2563 manager.go:216] Machine: {Timestamp:2026-07-29 12:25:27.426766009 +0000 UTC m=+0.329962676 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3205332 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d07957683e0b58e8bc4233ac29a6f SystemUUID:ec2d0795-7683-e0b5-8e8b-c4233ac29a6f BootID:d10a2672-24d6-4269-a006-d2e1439723c4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:6c:07:e5:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:6c:07:e5:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:2a:f7:07:ab:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 12:25:27.428013 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.428011 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 12:25:27.428111 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.428076 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 12:25:27.430694 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.430672 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 12:25:27.430825 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.430697 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-159.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 12:25:27.431408 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.431391 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 12:25:27.431448 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.431420 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 12:25:27.431448 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.431428 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 12:25:27.431448 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.431436 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 12:25:27.432147 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.432135 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 12:25:27.433307 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.433295 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 12:25:27.436072 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436061 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 29 12:25:27.436179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436170 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 12:25:27.436211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436196 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 29 12:25:27.436211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436205 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 12:25:27.436267 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436216 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 12:25:27.436267 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436224 2563 kubelet.go:398] "Adding apiserver pod source" Jul 29 12:25:27.436267 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436245 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 12:25:27.436664 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436651 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 12:25:27.436703 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.436675 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 12:25:27.438790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.438775 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 12:25:27.440105 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440093 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 12:25:27.440145 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440121 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 12:25:27.440642 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440632 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 12:25:27.440671 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440647 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 12:25:27.440671 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440653 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 12:25:27.440671 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440658 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 12:25:27.440750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440676 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 12:25:27.440750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440685 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 12:25:27.440882 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440874 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 12:25:27.440911 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440882 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 12:25:27.440911 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440888 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 12:25:27.440911 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440893 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 12:25:27.440911 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440901 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 12:25:27.440911 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.440908 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 12:25:27.441635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.441625 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 12:25:27.441635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.441635 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 12:25:27.445046 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.445031 2563 server.go:1268] "Started kubelet" Jul 29 12:25:27.445187 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.445167 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 12:25:27.445244 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.445133 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 12:25:27.445298 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.445266 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 12:25:27.445879 ip-10-0-137-159 systemd[1]: Started Kubernetes Kubelet. Jul 29 12:25:27.447599 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.447585 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 29 12:25:27.449864 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.449844 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 12:25:27.450049 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.450033 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-159.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 12:25:27.450421 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.450396 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 12:25:27.450421 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.450396 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 12:25:27.450992 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.450972 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 12:25:27.453353 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.452454 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-159.ec2.internal.18c6c24a7af1964a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-159.ec2.internal,UID:ip-10-0-137-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-159.ec2.internal,},FirstTimestamp:2026-07-29 12:25:27.445001802 +0000 UTC m=+0.348198469,LastTimestamp:2026-07-29 12:25:27.445001802 +0000 UTC m=+0.348198469,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-159.ec2.internal,}" Jul 29 12:25:27.455846 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.455829 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 12:25:27.455846 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.455829 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 12:25:27.455971 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.455933 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:27.456396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456374 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 12:25:27.456396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456375 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 12:25:27.456531 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456412 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 12:25:27.456531 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456497 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 29 12:25:27.456531 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456505 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 29 12:25:27.456778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456763 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 12:25:27.456778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456777 2563 factory.go:55] Registering systemd factory Jul 29 12:25:27.456886 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456786 2563 factory.go:223] Registration of the systemd container factory successfully Jul 29 12:25:27.456972 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456958 2563 factory.go:153] Registering CRI-O factory Jul 29 12:25:27.456972 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456974 2563 factory.go:223] Registration of the crio container factory successfully Jul 29 12:25:27.457108 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.456995 2563 factory.go:103] Registering Raw factory Jul 29 12:25:27.457108 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.457010 2563 manager.go:1196] Started watching for new ooms in manager Jul 29 12:25:27.457386 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.457374 2563 manager.go:349] Starting recovery of all containers Jul 29 12:25:27.461998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.461976 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pt782" Jul 29 12:25:27.463825 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.463797 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 12:25:27.463825 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.463817 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-159.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 12:25:27.467267 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.467252 2563 manager.go:354] Recovery completed Jul 29 12:25:27.470807 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.470791 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pt782" Jul 29 12:25:27.471535 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.471523 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.473472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.473440 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.473559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.473488 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.473559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.473509 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.474292 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.474272 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 12:25:27.474292 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.474291 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 12:25:27.474413 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.474309 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 29 12:25:27.474413 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.474399 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.475333 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.475313 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.475406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.475346 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.475406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.475360 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.475516 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.475505 2563 policy_none.go:49] "None policy: Start" Jul 29 12:25:27.475566 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.475519 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 12:25:27.475566 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.475529 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 12:25:27.476180 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.476085 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-159.ec2.internal.18c6c24a7ca4097d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-159.ec2.internal,UID:ip-10-0-137-159.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-159.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-159.ec2.internal,},FirstTimestamp:2026-07-29 12:25:27.473473917 +0000 UTC m=+0.376670588,LastTimestamp:2026-07-29 12:25:27.473473917 +0000 UTC m=+0.376670588,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-159.ec2.internal,}" Jul 29 12:25:27.476519 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.476506 2563 policy_none.go:47] "Start" Jul 29 12:25:27.518007 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.517993 2563 manager.go:337] "Starting Device Plugin manager" Jul 29 12:25:27.518081 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.518024 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 12:25:27.518081 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.518036 2563 server.go:87] "Starting device plugin registration server" Jul 29 12:25:27.518284 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.518273 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 12:25:27.518331 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.518285 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 12:25:27.518391 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.518377 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 12:25:27.518491 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.518470 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 12:25:27.518491 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.518482 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 12:25:27.518963 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.518943 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 12:25:27.519055 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.518987 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:27.618801 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.618778 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.619821 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.619664 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.619821 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.619693 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.619821 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.619706 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.619821 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.619729 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.627775 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.627753 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.627775 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.627774 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-159.ec2.internal\": node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:27.643912 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.643861 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:27.650918 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.650892 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 12:25:27.651985 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.651962 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 12:25:27.652048 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.652008 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 12:25:27.652048 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.652031 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 12:25:27.652104 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.652069 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 12:25:27.654500 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.654483 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 12:25:27.744051 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.744033 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:27.752132 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.752109 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal"] Jul 29 12:25:27.752180 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.752171 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.753569 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.753555 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.753633 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.753582 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.753633 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.753593 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.754803 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.754791 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.754919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.754898 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.754960 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.754937 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.755512 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.755491 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.755591 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.755517 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.755591 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.755527 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.755591 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.755494 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.755706 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.755598 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.755706 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.755613 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.756891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.756876 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.756939 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.756907 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 12:25:27.757135 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.757119 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/811ff241dbd55c6efd8d71a1c457d08b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal\" (UID: \"811ff241dbd55c6efd8d71a1c457d08b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.757173 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.757144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/811ff241dbd55c6efd8d71a1c457d08b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal\" (UID: \"811ff241dbd55c6efd8d71a1c457d08b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.757173 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.757158 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7b4005b800080bf2e43d7f05963a9c0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-159.ec2.internal\" (UID: \"a7b4005b800080bf2e43d7f05963a9c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.757627 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.757605 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientMemory" Jul 29 12:25:27.757690 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.757636 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 12:25:27.757690 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.757650 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeHasSufficientPID" Jul 29 12:25:27.786956 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.786934 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-159.ec2.internal\" not found" node="ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.790943 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.790929 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-159.ec2.internal\" not found" node="ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.844235 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.844219 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:27.858219 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.858201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/811ff241dbd55c6efd8d71a1c457d08b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal\" (UID: \"811ff241dbd55c6efd8d71a1c457d08b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.858294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.858223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/811ff241dbd55c6efd8d71a1c457d08b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal\" (UID: \"811ff241dbd55c6efd8d71a1c457d08b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.858294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.858240 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7b4005b800080bf2e43d7f05963a9c0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-159.ec2.internal\" (UID: \"a7b4005b800080bf2e43d7f05963a9c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.858294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.858279 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7b4005b800080bf2e43d7f05963a9c0-config\") pod \"kube-apiserver-proxy-ip-10-0-137-159.ec2.internal\" (UID: \"a7b4005b800080bf2e43d7f05963a9c0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.858294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.858292 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/811ff241dbd55c6efd8d71a1c457d08b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal\" (UID: \"811ff241dbd55c6efd8d71a1c457d08b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.858424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:27.858309 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/811ff241dbd55c6efd8d71a1c457d08b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal\" (UID: \"811ff241dbd55c6efd8d71a1c457d08b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:27.944933 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:27.944891 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.045296 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.045267 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.089703 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.089683 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:28.094815 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.094726 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" Jul 29 12:25:28.145575 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.145556 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.245970 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.245925 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.302852 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.302827 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 12:25:28.346638 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.346617 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.400002 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.399989 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 12:25:28.400378 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.400091 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 12:25:28.400378 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.400148 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 12:25:28.447414 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.447395 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.475281 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.475257 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 12:20:27 +0000 UTC" deadline="2028-01-13 11:10:11.752481624 +0000 UTC" Jul 29 12:25:28.475281 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.475280 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12790h44m43.277204007s" Jul 29 12:25:28.475717 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.475703 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 12:25:28.547805 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.547751 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.601328 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:28.601303 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811ff241dbd55c6efd8d71a1c457d08b.slice/crio-12ce111956d34ad41f32b63c09639cc87faa4b433c953360d596281fc34692e4 WatchSource:0}: Error finding container 12ce111956d34ad41f32b63c09639cc87faa4b433c953360d596281fc34692e4: Status 404 returned error can't find the container with id 12ce111956d34ad41f32b63c09639cc87faa4b433c953360d596281fc34692e4 Jul 29 12:25:28.601653 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:28.601635 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b4005b800080bf2e43d7f05963a9c0.slice/crio-6fc686c8d99504a220af1ef68bdf2536a8e1db6d99bd413399f50790993e92df WatchSource:0}: Error finding container 6fc686c8d99504a220af1ef68bdf2536a8e1db6d99bd413399f50790993e92df: Status 404 returned error can't find the container with id 6fc686c8d99504a220af1ef68bdf2536a8e1db6d99bd413399f50790993e92df Jul 29 12:25:28.607875 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.607861 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 12:25:28.648699 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.648675 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.654382 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.654334 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" event={"ID":"811ff241dbd55c6efd8d71a1c457d08b","Type":"ContainerStarted","Data":"12ce111956d34ad41f32b63c09639cc87faa4b433c953360d596281fc34692e4"} Jul 29 12:25:28.655183 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.655165 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" event={"ID":"a7b4005b800080bf2e43d7f05963a9c0","Type":"ContainerStarted","Data":"6fc686c8d99504a220af1ef68bdf2536a8e1db6d99bd413399f50790993e92df"} Jul 29 12:25:28.749567 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.749546 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.849892 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:28.849847 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-159.ec2.internal\" not found" Jul 29 12:25:28.946437 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.946417 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 12:25:28.956955 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.956941 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" Jul 29 12:25:28.967261 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.967248 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 12:25:28.968062 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.968050 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" Jul 29 12:25:28.978269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:28.978254 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 12:25:29.437994 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.437973 2563 apiserver.go:52] "Watching apiserver" Jul 29 12:25:29.445693 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.445665 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 12:25:29.446373 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.446106 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-xm56m","openshift-image-registry/node-ca-9svn6","openshift-multus/multus-additional-cni-plugins-6vqb2","openshift-network-diagnostics/network-check-target-7m5ln","openshift-network-operator/iptables-alerter-jxcrs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7","openshift-cluster-node-tuning-operator/tuned-mhnbm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal","openshift-multus/multus-x4qbv","openshift-multus/network-metrics-daemon-zcf8c","openshift-ovn-kubernetes/ovnkube-node-xvhcg","kube-system/global-pull-secret-syncer-lpk5s","kube-system/konnectivity-agent-cxphl","kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal"] Jul 29 12:25:29.448550 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.448486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.450650 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.450617 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.451358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.450865 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m9b7r\"" Jul 29 12:25:29.451358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.451088 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.451358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.451165 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.452932 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.452663 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.452932 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.452789 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7snzf\"" Jul 29 12:25:29.453317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.453174 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 12:25:29.453317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.453230 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 12:25:29.455006 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.454787 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:29.455006 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.454869 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:29.455186 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.455166 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.455344 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.455274 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 12:25:29.455975 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.455560 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kw8fz\"" Jul 29 12:25:29.455975 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.455761 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 12:25:29.455975 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.455797 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.455975 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.455962 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 12:25:29.456179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.456070 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 12:25:29.456955 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.456937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.459555 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.459531 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.460795 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.459649 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.460795 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.460049 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l9k6k\"" Jul 29 12:25:29.461137 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.461115 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.461579 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.461559 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 12:25:29.462165 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.461992 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.462653 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.462485 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7b4rm\"" Jul 29 12:25:29.462653 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.462576 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.463398 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.463382 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 12:25:29.464175 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.464158 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 12:25:29.464373 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.464359 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xkk2l\"" Jul 29 12:25:29.464876 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.464414 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.464952 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.464426 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.465856 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ae06bbe-8fd9-437c-b8b0-44e5110c9db5-konnectivity-ca\") pod \"konnectivity-agent-cxphl\" (UID: \"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5\") " pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.465943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465873 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61c35865-9c6a-439e-9a01-591169e965c6-host-slash\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.465943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-os-release\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.465943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465904 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-modprobe-d\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.465943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-system-cni-dir\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.465943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-systemd\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465950 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-sys\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465963 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5422892-facc-4b16-b1fe-49e9ab976f17-tmp\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ae06bbe-8fd9-437c-b8b0-44e5110c9db5-agent-certs\") pod \"konnectivity-agent-cxphl\" (UID: \"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5\") " pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.465990 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466039 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/61c35865-9c6a-439e-9a01-591169e965c6-iptables-alerter-script\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466059 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-kubernetes\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466098 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snq2f\" (UniqueName: \"kubernetes.io/projected/c5422892-facc-4b16-b1fe-49e9ab976f17-kube-api-access-snq2f\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466146 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.466194 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl588\" (UniqueName: \"kubernetes.io/projected/c9942c8c-5762-439f-8ce1-8bd261960a9e-kube-api-access-nl588\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysctl-d\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466239 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysctl-conf\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466267 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-var-lib-kubelet\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466288 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-cnibin\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466308 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466381 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gm95\" (UniqueName: \"kubernetes.io/projected/61c35865-9c6a-439e-9a01-591169e965c6-kube-api-access-8gm95\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysconfig\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466433 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-run\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466449 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-lib-modules\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-host\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466499 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-tuned\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.466750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.466703 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.467967 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.467756 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.468076 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.467775 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.468396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.467896 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 12:25:29.468396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.467932 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5mkfx\"" Jul 29 12:25:29.468640 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.468623 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:29.468740 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.468701 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:29.469833 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.469806 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 12:25:29.469995 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.469982 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.470204 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.470172 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.470489 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.470445 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fpkns\"" Jul 29 12:25:29.470677 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.470662 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.472950 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.472884 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.473032 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.472934 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473306 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473327 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473363 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473306 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pc2w9\"" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473502 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473574 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 12:25:29.473707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.473625 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 12:25:29.491880 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.491852 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gxrjp" Jul 29 12:25:29.500339 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.500321 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gxrjp" Jul 29 12:25:29.557242 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.557222 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 12:25:29.566733 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566711 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ae06bbe-8fd9-437c-b8b0-44e5110c9db5-konnectivity-ca\") pod \"konnectivity-agent-cxphl\" (UID: \"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5\") " pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.566822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61c35865-9c6a-439e-9a01-591169e965c6-host-slash\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.566822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566765 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-etc-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.566822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566787 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-netns\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.566822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566808 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbdjj\" (UniqueName: \"kubernetes.io/projected/bf5272ab-7183-4547-90c2-197987a1e3c1-kube-api-access-kbdjj\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566829 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-cni-multus\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566849 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-hostroot\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnst\" (UniqueName: \"kubernetes.io/projected/6d292488-7717-4073-9f75-5c47539006d6-kube-api-access-dxnst\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61c35865-9c6a-439e-9a01-591169e965c6-host-slash\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvx5t\" (UniqueName: \"kubernetes.io/projected/700bc436-2f38-40e7-a91d-ac67dad37ed0-kube-api-access-kvx5t\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-modprobe-d\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.566998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.566988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-system-cni-dir\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567008 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-systemd-units\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567034 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-run-netns\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567054 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-cni-bin\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567066 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-system-cni-dir\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567077 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-ovnkube-config\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567097 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-modprobe-d\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567105 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1972259-e332-4fbb-b3db-247f6bdb3f95-dbus\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567130 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw4fj\" (UniqueName: \"kubernetes.io/projected/7e324f8b-44ff-4145-9926-4078115e22cb-kube-api-access-fw4fj\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567201 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567223 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-cni-netd\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567243 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-cnibin\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d292488-7717-4073-9f75-5c47539006d6-cni-binary-copy\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-sys\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ae06bbe-8fd9-437c-b8b0-44e5110c9db5-agent-certs\") pod \"konnectivity-agent-cxphl\" (UID: \"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5\") " pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567372 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-sys\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567404 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ae06bbe-8fd9-437c-b8b0-44e5110c9db5-konnectivity-ca\") pod \"konnectivity-agent-cxphl\" (UID: \"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5\") " pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567447 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/61c35865-9c6a-439e-9a01-591169e965c6-iptables-alerter-script\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567502 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-kubelet\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-slash\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567548 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567569 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-socket-dir-parent\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snq2f\" (UniqueName: \"kubernetes.io/projected/c5422892-facc-4b16-b1fe-49e9ab976f17-kube-api-access-snq2f\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567628 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl588\" (UniqueName: \"kubernetes.io/projected/c9942c8c-5762-439f-8ce1-8bd261960a9e-kube-api-access-nl588\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.567700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567684 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-etc-kubernetes\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-socket-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567727 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysctl-conf\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567748 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf5272ab-7183-4547-90c2-197987a1e3c1-ovn-node-metrics-cert\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567768 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-kubelet\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhlgj\" (UniqueName: \"kubernetes.io/projected/7d7e341b-4606-4c63-a278-9f9d71a68296-kube-api-access-hhlgj\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567812 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567823 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgmmw\" (UniqueName: \"kubernetes.io/projected/77e9deb9-c878-4140-b92a-c1ffeaa0abff-kube-api-access-mgmmw\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567859 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-tuned\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567883 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e324f8b-44ff-4145-9926-4078115e22cb-host\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-os-release\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567932 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/61c35865-9c6a-439e-9a01-591169e965c6-iptables-alerter-script\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysctl-conf\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567945 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-systemd\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567976 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.568472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.567983 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568013 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-os-release\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568060 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-registration-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568091 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/700bc436-2f38-40e7-a91d-ac67dad37ed0-hosts-file\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568110 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1972259-e332-4fbb-b3db-247f6bdb3f95-kubelet-config\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568138 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-cni-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568184 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-device-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568213 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-sys-fs\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-var-lib-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568283 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-ovnkube-script-lib\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568299 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-cni-bin\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568315 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-systemd\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e324f8b-44ff-4145-9926-4078115e22cb-serviceca\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-log-socket\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568376 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-systemd\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-conf-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568442 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/700bc436-2f38-40e7-a91d-ac67dad37ed0-tmp-dir\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-env-overrides\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5422892-facc-4b16-b1fe-49e9ab976f17-tmp\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568583 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-ovn\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568603 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-os-release\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568628 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-k8s-cni-cncf-io\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568648 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-multus-certs\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-kubernetes\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568725 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-node-log\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568765 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-kubernetes\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568777 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysctl-d\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-var-lib-kubelet\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568869 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-cnibin\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.569837 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568882 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysctl-d\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gm95\" (UniqueName: \"kubernetes.io/projected/61c35865-9c6a-439e-9a01-591169e965c6-kube-api-access-8gm95\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568955 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-system-cni-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d292488-7717-4073-9f75-5c47539006d6-multus-daemon-config\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.568985 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-var-lib-kubelet\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysconfig\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569021 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9942c8c-5762-439f-8ce1-8bd261960a9e-cnibin\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-run\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569084 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-lib-modules\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569108 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-sysconfig\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569129 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-host\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569192 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-run\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569215 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-lib-modules\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569216 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5422892-facc-4b16-b1fe-49e9ab976f17-host\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569346 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.570561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.569401 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9942c8c-5762-439f-8ce1-8bd261960a9e-cni-binary-copy\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.571417 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.571357 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5422892-facc-4b16-b1fe-49e9ab976f17-etc-tuned\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.571675 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.571657 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ae06bbe-8fd9-437c-b8b0-44e5110c9db5-agent-certs\") pod \"konnectivity-agent-cxphl\" (UID: \"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5\") " pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.572648 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.572608 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5422892-facc-4b16-b1fe-49e9ab976f17-tmp\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.575674 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.575651 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:25:29.575674 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.575672 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:25:29.575813 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.575683 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:29.575813 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.575744 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:25:30.075726776 +0000 UTC m=+2.978923441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:29.578295 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.578248 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gm95\" (UniqueName: \"kubernetes.io/projected/61c35865-9c6a-439e-9a01-591169e965c6-kube-api-access-8gm95\") pod \"iptables-alerter-jxcrs\" (UID: \"61c35865-9c6a-439e-9a01-591169e965c6\") " pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.578846 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.578811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snq2f\" (UniqueName: \"kubernetes.io/projected/c5422892-facc-4b16-b1fe-49e9ab976f17-kube-api-access-snq2f\") pod \"tuned-mhnbm\" (UID: \"c5422892-facc-4b16-b1fe-49e9ab976f17\") " pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.578950 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.578927 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl588\" (UniqueName: \"kubernetes.io/projected/c9942c8c-5762-439f-8ce1-8bd261960a9e-kube-api-access-nl588\") pod \"multus-additional-cni-plugins-6vqb2\" (UID: \"c9942c8c-5762-439f-8ce1-8bd261960a9e\") " pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.669574 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669547 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1972259-e332-4fbb-b3db-247f6bdb3f95-kubelet-config\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.669705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-cni-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.669705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-device-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.669705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-sys-fs\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.669705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669660 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a1972259-e332-4fbb-b3db-247f6bdb3f95-kubelet-config\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.669705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669679 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-var-lib-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.669705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669700 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.669948 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669720 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-device-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.669948 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-ovnkube-script-lib\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.670397 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.669720 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-sys-fs\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.670397 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.670379 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-cni-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.670907 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-cni-bin\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.670972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e324f8b-44ff-4145-9926-4078115e22cb-serviceca\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-log-socket\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671019 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-ovnkube-script-lib\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671087 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-cni-bin\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-conf-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671133 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-var-lib-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/700bc436-2f38-40e7-a91d-ac67dad37ed0-tmp-dir\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671161 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671195 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-conf-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-env-overrides\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671228 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-ovn\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-os-release\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-k8s-cni-cncf-io\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671312 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-multus-certs\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-node-log\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671378 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-system-cni-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d292488-7717-4073-9f75-5c47539006d6-multus-daemon-config\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-etc-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671507 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7e324f8b-44ff-4145-9926-4078115e22cb-serviceca\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671524 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-netns\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671553 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbdjj\" (UniqueName: \"kubernetes.io/projected/bf5272ab-7183-4547-90c2-197987a1e3c1-kube-api-access-kbdjj\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-log-socket\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-cni-multus\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671622 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-k8s-cni-cncf-io\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671626 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-cni-multus\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671707 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-hostroot\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671735 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnst\" (UniqueName: \"kubernetes.io/projected/6d292488-7717-4073-9f75-5c47539006d6-kube-api-access-dxnst\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671781 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvx5t\" (UniqueName: \"kubernetes.io/projected/700bc436-2f38-40e7-a91d-ac67dad37ed0-kube-api-access-kvx5t\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671792 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/700bc436-2f38-40e7-a91d-ac67dad37ed0-tmp-dir\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671833 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671833 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-systemd-units\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-run-netns\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671920 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-run-netns\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.673893 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.671985 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-hostroot\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672056 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-multus-certs\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672238 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-node-log\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672244 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-os-release\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672281 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-system-cni-dir\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672332 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-systemd-units\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672530 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-cni-bin\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672603 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-ovn\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672613 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-ovnkube-config\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672690 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-cni-bin\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672726 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-run-netns\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672582 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-etc-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672751 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-env-overrides\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672759 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d292488-7717-4073-9f75-5c47539006d6-multus-daemon-config\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1972259-e332-4fbb-b3db-247f6bdb3f95-dbus\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672822 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a1972259-e332-4fbb-b3db-247f6bdb3f95-dbus\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw4fj\" (UniqueName: \"kubernetes.io/projected/7e324f8b-44ff-4145-9926-4078115e22cb-kube-api-access-fw4fj\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.674665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672906 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-openvswitch\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672906 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-cni-netd\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672941 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-cni-netd\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672944 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-cnibin\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d292488-7717-4073-9f75-5c47539006d6-cni-binary-copy\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.672986 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-cnibin\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf5272ab-7183-4547-90c2-197987a1e3c1-ovnkube-config\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673009 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-kubelet\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673005 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673052 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-slash\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673083 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-slash\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-socket-dir-parent\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673158 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-etc-kubernetes\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-socket-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673243 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-host-kubelet\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673243 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-multus-socket-dir-parent\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.675321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf5272ab-7183-4547-90c2-197987a1e3c1-ovn-node-metrics-cert\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-kubelet\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhlgj\" (UniqueName: \"kubernetes.io/projected/7d7e341b-4606-4c63-a278-9f9d71a68296-kube-api-access-hhlgj\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673364 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-socket-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.673161 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.673510 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:30.173476639 +0000 UTC m=+3.076673307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d292488-7717-4073-9f75-5c47539006d6-cni-binary-copy\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673695 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-host-var-lib-kubelet\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673741 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d292488-7717-4073-9f75-5c47539006d6-etc-kubernetes\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673773 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgmmw\" (UniqueName: \"kubernetes.io/projected/77e9deb9-c878-4140-b92a-c1ffeaa0abff-kube-api-access-mgmmw\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e324f8b-44ff-4145-9926-4078115e22cb-host\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-systemd\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673914 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673912 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7e324f8b-44ff-4145-9926-4078115e22cb-host\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673951 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf5272ab-7183-4547-90c2-197987a1e3c1-run-systemd\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.676061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.673981 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-registration-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.676800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.674012 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/700bc436-2f38-40e7-a91d-ac67dad37ed0-hosts-file\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.676800 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.674026 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:29.676800 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:29.674082 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:30.174066641 +0000 UTC m=+3.077263306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:29.676800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.674082 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/700bc436-2f38-40e7-a91d-ac67dad37ed0-hosts-file\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.676800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.674101 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/77e9deb9-c878-4140-b92a-c1ffeaa0abff-registration-dir\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.676800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.675787 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf5272ab-7183-4547-90c2-197987a1e3c1-ovn-node-metrics-cert\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.690002 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.689935 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnst\" (UniqueName: \"kubernetes.io/projected/6d292488-7717-4073-9f75-5c47539006d6-kube-api-access-dxnst\") pod \"multus-x4qbv\" (UID: \"6d292488-7717-4073-9f75-5c47539006d6\") " pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.690002 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.689980 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvx5t\" (UniqueName: \"kubernetes.io/projected/700bc436-2f38-40e7-a91d-ac67dad37ed0-kube-api-access-kvx5t\") pod \"node-resolver-xm56m\" (UID: \"700bc436-2f38-40e7-a91d-ac67dad37ed0\") " pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.691082 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.691047 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhlgj\" (UniqueName: \"kubernetes.io/projected/7d7e341b-4606-4c63-a278-9f9d71a68296-kube-api-access-hhlgj\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:29.691178 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.691160 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgmmw\" (UniqueName: \"kubernetes.io/projected/77e9deb9-c878-4140-b92a-c1ffeaa0abff-kube-api-access-mgmmw\") pod \"aws-ebs-csi-driver-node-nhpd7\" (UID: \"77e9deb9-c878-4140-b92a-c1ffeaa0abff\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.691264 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.691248 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbdjj\" (UniqueName: \"kubernetes.io/projected/bf5272ab-7183-4547-90c2-197987a1e3c1-kube-api-access-kbdjj\") pod \"ovnkube-node-xvhcg\" (UID: \"bf5272ab-7183-4547-90c2-197987a1e3c1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:29.691666 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.691648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw4fj\" (UniqueName: \"kubernetes.io/projected/7e324f8b-44ff-4145-9926-4078115e22cb-kube-api-access-fw4fj\") pod \"node-ca-9svn6\" (UID: \"7e324f8b-44ff-4145-9926-4078115e22cb\") " pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.715640 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.715618 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 12:25:29.764408 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.764394 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" Jul 29 12:25:29.775801 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.775782 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:29.784015 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.783994 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" Jul 29 12:25:29.791061 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.791045 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jxcrs" Jul 29 12:25:29.798552 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.798537 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" Jul 29 12:25:29.804868 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.804853 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x4qbv" Jul 29 12:25:29.812212 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.812198 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xm56m" Jul 29 12:25:29.819515 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.819502 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9svn6" Jul 29 12:25:29.825894 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:29.825881 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:30.077295 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.077223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:30.077436 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.077352 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:25:30.077436 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.077368 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:25:30.077436 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.077379 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:30.077582 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.077441 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:25:31.077416459 +0000 UTC m=+3.980613125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:30.097067 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.097049 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 12:25:30.178252 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.178225 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:30.178385 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.178269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:30.178385 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.178377 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:30.178510 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.178381 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:30.178510 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.178442 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:31.178417528 +0000 UTC m=+4.081614195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:30.178510 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.178474 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:31.178450329 +0000 UTC m=+4.081646986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:30.411942 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.411920 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5422892_facc_4b16_b1fe_49e9ab976f17.slice/crio-d1cbbfb6183080bb9d97bbbb29fdc0f4d627963f3576fecce185d4c98eb85adb WatchSource:0}: Error finding container d1cbbfb6183080bb9d97bbbb29fdc0f4d627963f3576fecce185d4c98eb85adb: Status 404 returned error can't find the container with id d1cbbfb6183080bb9d97bbbb29fdc0f4d627963f3576fecce185d4c98eb85adb Jul 29 12:25:30.413779 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.413695 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf5272ab_7183_4547_90c2_197987a1e3c1.slice/crio-68bff0355bf8c301a3c8883eea7204d7bf9cf6ca1d8f013468649fbe911d449d WatchSource:0}: Error finding container 68bff0355bf8c301a3c8883eea7204d7bf9cf6ca1d8f013468649fbe911d449d: Status 404 returned error can't find the container with id 68bff0355bf8c301a3c8883eea7204d7bf9cf6ca1d8f013468649fbe911d449d Jul 29 12:25:30.416012 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.415989 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700bc436_2f38_40e7_a91d_ac67dad37ed0.slice/crio-30a618e42bcf1cbb649dc5e1f69bcf574184ff3471ad44d0b446e85cde348ba8 WatchSource:0}: Error finding container 30a618e42bcf1cbb649dc5e1f69bcf574184ff3471ad44d0b446e85cde348ba8: Status 404 returned error can't find the container with id 30a618e42bcf1cbb649dc5e1f69bcf574184ff3471ad44d0b446e85cde348ba8 Jul 29 12:25:30.416793 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.416770 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e324f8b_44ff_4145_9926_4078115e22cb.slice/crio-fa3a4b9f7ac1ae0f14e65b441a2929b4c18ace1c451ac9dcb74f1bbcf28cbf55 WatchSource:0}: Error finding container fa3a4b9f7ac1ae0f14e65b441a2929b4c18ace1c451ac9dcb74f1bbcf28cbf55: Status 404 returned error can't find the container with id fa3a4b9f7ac1ae0f14e65b441a2929b4c18ace1c451ac9dcb74f1bbcf28cbf55 Jul 29 12:25:30.417537 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.417482 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d292488_7717_4073_9f75_5c47539006d6.slice/crio-6e00af08974acd41ff8394fe5d0b78f5c4f5e1180e19d8a8c891da616f359417 WatchSource:0}: Error finding container 6e00af08974acd41ff8394fe5d0b78f5c4f5e1180e19d8a8c891da616f359417: Status 404 returned error can't find the container with id 6e00af08974acd41ff8394fe5d0b78f5c4f5e1180e19d8a8c891da616f359417 Jul 29 12:25:30.421622 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.421470 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77e9deb9_c878_4140_b92a_c1ffeaa0abff.slice/crio-c4fb0604b69b200125d1d31e9d50fbb9be28b6a4ff1ab7f9a59be83fa869aaa1 WatchSource:0}: Error finding container c4fb0604b69b200125d1d31e9d50fbb9be28b6a4ff1ab7f9a59be83fa869aaa1: Status 404 returned error can't find the container with id c4fb0604b69b200125d1d31e9d50fbb9be28b6a4ff1ab7f9a59be83fa869aaa1 Jul 29 12:25:30.422869 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:25:30.422809 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9942c8c_5762_439f_8ce1_8bd261960a9e.slice/crio-4db672b170654213fed98f650cd1e5dbbc59ae7666c21e014a6e051a771706e2 WatchSource:0}: Error finding container 4db672b170654213fed98f650cd1e5dbbc59ae7666c21e014a6e051a771706e2: Status 404 returned error can't find the container with id 4db672b170654213fed98f650cd1e5dbbc59ae7666c21e014a6e051a771706e2 Jul 29 12:25:30.505026 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.504978 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 12:20:29 +0000 UTC" deadline="2028-05-09 05:20:01.464708918 +0000 UTC" Jul 29 12:25:30.505373 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.505027 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15592h54m30.959684452s" Jul 29 12:25:30.653242 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.653175 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:30.653325 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:30.653290 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:30.659731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.659707 2563 generic.go:356] "Generic (PLEG): container finished" podID="811ff241dbd55c6efd8d71a1c457d08b" containerID="439b2a9861b7508d3875e640478f73fbba9d0ee67967efd2ee299bbe8cec3e36" exitCode=0 Jul 29 12:25:30.659814 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.659731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" event={"ID":"811ff241dbd55c6efd8d71a1c457d08b","Type":"ContainerDied","Data":"439b2a9861b7508d3875e640478f73fbba9d0ee67967efd2ee299bbe8cec3e36"} Jul 29 12:25:30.660800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.660778 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" event={"ID":"77e9deb9-c878-4140-b92a-c1ffeaa0abff","Type":"ContainerStarted","Data":"c4fb0604b69b200125d1d31e9d50fbb9be28b6a4ff1ab7f9a59be83fa869aaa1"} Jul 29 12:25:30.661726 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.661705 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cxphl" event={"ID":"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5","Type":"ContainerStarted","Data":"ea759978371f974f7e3ad2ab48a8473bd6ac54d7e1f1e4d87feeb7137c2d2450"} Jul 29 12:25:30.662642 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.662618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xm56m" event={"ID":"700bc436-2f38-40e7-a91d-ac67dad37ed0","Type":"ContainerStarted","Data":"30a618e42bcf1cbb649dc5e1f69bcf574184ff3471ad44d0b446e85cde348ba8"} Jul 29 12:25:30.663703 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.663682 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"68bff0355bf8c301a3c8883eea7204d7bf9cf6ca1d8f013468649fbe911d449d"} Jul 29 12:25:30.664678 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.664652 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" event={"ID":"c5422892-facc-4b16-b1fe-49e9ab976f17","Type":"ContainerStarted","Data":"d1cbbfb6183080bb9d97bbbb29fdc0f4d627963f3576fecce185d4c98eb85adb"} Jul 29 12:25:30.665972 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.665946 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" event={"ID":"a7b4005b800080bf2e43d7f05963a9c0","Type":"ContainerStarted","Data":"c89c02ac92475bb37ed6a13cb3e5237de6c4e63296b90d3b086297336a64aae6"} Jul 29 12:25:30.666971 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.666956 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerStarted","Data":"4db672b170654213fed98f650cd1e5dbbc59ae7666c21e014a6e051a771706e2"} Jul 29 12:25:30.668051 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.668034 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jxcrs" event={"ID":"61c35865-9c6a-439e-9a01-591169e965c6","Type":"ContainerStarted","Data":"7538185ce45d4b88e42dc20b11176945e630e52bfa36e2b6ae8e060ecaad3714"} Jul 29 12:25:30.669007 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.668991 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x4qbv" event={"ID":"6d292488-7717-4073-9f75-5c47539006d6","Type":"ContainerStarted","Data":"6e00af08974acd41ff8394fe5d0b78f5c4f5e1180e19d8a8c891da616f359417"} Jul 29 12:25:30.669982 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:30.669959 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9svn6" event={"ID":"7e324f8b-44ff-4145-9926-4078115e22cb","Type":"ContainerStarted","Data":"fa3a4b9f7ac1ae0f14e65b441a2929b4c18ace1c451ac9dcb74f1bbcf28cbf55"} Jul 29 12:25:31.085362 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.085275 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:31.085561 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.085444 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:25:31.085561 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.085483 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:25:31.085561 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.085498 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:31.085561 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.085554 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:25:33.085534015 +0000 UTC m=+5.988730676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:31.186148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.186113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:31.186333 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.186170 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:31.186333 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.186306 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:31.186441 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.186386 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:33.186349159 +0000 UTC m=+6.089545819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:31.186905 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.186884 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:31.187007 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.186941 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:33.186926273 +0000 UTC m=+6.090122939 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:31.505824 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.505716 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 12:20:29 +0000 UTC" deadline="2028-02-07 16:30:11.779452321 +0000 UTC" Jul 29 12:25:31.505824 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.505771 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13396h4m40.273685364s" Jul 29 12:25:31.653344 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.653313 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:31.653523 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.653451 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:31.653978 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.653958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:31.654139 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:31.654107 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:31.706638 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.706612 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" event={"ID":"811ff241dbd55c6efd8d71a1c457d08b","Type":"ContainerStarted","Data":"17a98732a1a12c2b60ae5164224223e3ab896b8ff70c9ac6a064ec9c009c8947"} Jul 29 12:25:31.723297 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:31.723244 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-159.ec2.internal" podStartSLOduration=3.723226713 podStartE2EDuration="3.723226713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:30.691208757 +0000 UTC m=+3.594405430" watchObservedRunningTime="2026-07-29 12:25:31.723226713 +0000 UTC m=+4.626423391" Jul 29 12:25:32.652887 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:32.652856 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:32.653326 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:32.653015 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:33.099755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:33.099672 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:33.099906 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.099879 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:25:33.099906 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.099895 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:25:33.099906 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.099907 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:33.100095 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.099963 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:25:37.099946819 +0000 UTC m=+10.003143486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:33.200698 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:33.200667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:33.200861 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:33.200754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:33.200920 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.200898 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:33.200968 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.200955 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:37.2009369 +0000 UTC m=+10.104133557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:33.201379 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.201360 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:33.201450 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.201412 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:37.201398135 +0000 UTC m=+10.104594790 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:33.653021 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:33.652990 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:33.653506 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.653126 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:33.653506 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:33.653265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:33.653506 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:33.653383 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:34.653008 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:34.652976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:34.653189 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:34.653108 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:35.652891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:35.652860 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:35.653046 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:35.652986 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:35.653524 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:35.653504 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:35.653907 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:35.653607 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:36.653053 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:36.653020 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:36.653215 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:36.653144 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:37.132390 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:37.132311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:37.132805 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.132479 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:25:37.132805 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.132497 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:25:37.132805 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.132508 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:37.132805 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.132555 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:25:45.132540298 +0000 UTC m=+18.035736952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:37.233244 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:37.233082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:37.233244 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:37.233153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:37.233244 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.233205 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:37.233244 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.233240 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:37.233688 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.233273 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:45.233253241 +0000 UTC m=+18.136449912 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:37.233688 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.233290 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:25:45.233280995 +0000 UTC m=+18.136477652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:37.653807 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:37.653355 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:37.653807 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.653482 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:37.654251 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:37.654108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:37.654251 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:37.654209 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:38.652362 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:38.652283 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:38.652795 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:38.652412 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:39.652314 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:39.652286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:39.652521 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:39.652286 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:39.652521 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:39.652406 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:39.652521 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:39.652485 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:40.652760 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:40.652731 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:40.653074 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:40.652826 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:41.652438 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:41.652408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:41.652647 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:41.652509 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:41.652647 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:41.652549 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:41.652647 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:41.652597 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:42.652616 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:42.652588 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:42.653056 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:42.652706 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:43.652999 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:43.652966 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:43.653487 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:43.653091 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:43.653487 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:43.653141 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:43.653487 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:43.653260 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:44.652599 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:44.652560 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:44.652792 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:44.652683 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:45.198212 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:45.198176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:45.198643 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.198311 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:25:45.198643 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.198331 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:25:45.198643 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.198343 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:45.198643 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.198395 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:26:01.198382981 +0000 UTC m=+34.101579636 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:25:45.298624 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:45.298591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:45.298757 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:45.298645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:45.298830 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.298752 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:45.298830 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.298775 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:45.298830 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.298819 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:01.298798148 +0000 UTC m=+34.201994819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:25:45.298976 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.298839 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:01.298828014 +0000 UTC m=+34.202024668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:25:45.652728 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:45.652659 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:45.652883 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.652766 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:45.652883 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:45.652825 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:45.652986 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:45.652944 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:46.652985 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:46.652955 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:46.653372 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:46.653060 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:47.653717 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:47.653687 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:47.654082 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:47.653778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:47.654082 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:47.653812 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:47.654082 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:47.653857 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:48.652512 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.652354 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:48.652654 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:48.652627 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:48.746545 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.746272 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" event={"ID":"c5422892-facc-4b16-b1fe-49e9ab976f17","Type":"ContainerStarted","Data":"02319a11d56772a3362174e11ce78ac47876a61799021adc1189e5f5b6aed33a"} Jul 29 12:25:48.748323 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.748298 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9942c8c-5762-439f-8ce1-8bd261960a9e" containerID="f180b21f3a6c998ef04dccc923fea50b3f1a1911797b844e68def7cbb70bacef" exitCode=0 Jul 29 12:25:48.748405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.748373 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerDied","Data":"f180b21f3a6c998ef04dccc923fea50b3f1a1911797b844e68def7cbb70bacef"} Jul 29 12:25:48.750059 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.750024 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x4qbv" event={"ID":"6d292488-7717-4073-9f75-5c47539006d6","Type":"ContainerStarted","Data":"abd250ea1ccb9c7cdee1764f02fe1e21d1905df2a7cc7217d6dc34657723784a"} Jul 29 12:25:48.751875 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.751855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9svn6" event={"ID":"7e324f8b-44ff-4145-9926-4078115e22cb","Type":"ContainerStarted","Data":"3f3f7885fc6247f2bd811ed3e3374cf4e57cee5b2506d71ca46cea73cbc3bdce"} Jul 29 12:25:48.753316 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.753291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" event={"ID":"77e9deb9-c878-4140-b92a-c1ffeaa0abff","Type":"ContainerStarted","Data":"3aab5cfd294bd7710a6afdded240de0283b880de947fbd878bea44036c38cfc8"} Jul 29 12:25:48.754721 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.754696 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cxphl" event={"ID":"6ae06bbe-8fd9-437c-b8b0-44e5110c9db5","Type":"ContainerStarted","Data":"b6139d407c226aeb7cd0880672fe17f800c6a806a00b8b17dea72a7619db218a"} Jul 29 12:25:48.756095 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.756076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xm56m" event={"ID":"700bc436-2f38-40e7-a91d-ac67dad37ed0","Type":"ContainerStarted","Data":"e3263d863066d86d932b357c11378ee34c081a101d4769b674d7c70614250008"} Jul 29 12:25:48.757974 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.757957 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"9fd89dfce0e9972c03a3ea5e7b6efc8a0bdd9b638468827886ff3df730b24bc3"} Jul 29 12:25:48.758052 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.758011 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"686f3fbaf9de82dcc78a68bb672a8e11d5a7439958ca02041a71caf78ea85093"} Jul 29 12:25:48.758052 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.758043 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"022ae7ff247d8e4bf7b2c9887631be1e6117c73dafd5b95c62c5471040f9b3e2"} Jul 29 12:25:48.767632 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.767596 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mhnbm" podStartSLOduration=3.8118605949999997 podStartE2EDuration="21.767584644s" totalImagesPullingTime="17.955724049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:48.767355071 +0000 UTC m=+21.670551746" watchObservedRunningTime="2026-07-29 12:25:48.767584644 +0000 UTC m=+21.670781319" Jul 29 12:25:48.767712 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.767681 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-159.ec2.internal" podStartSLOduration=20.767676739 podStartE2EDuration="20.767676739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:31.724117226 +0000 UTC m=+4.627313900" watchObservedRunningTime="2026-07-29 12:25:48.767676739 +0000 UTC m=+21.670873414" Jul 29 12:25:48.782865 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.782832 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cxphl" podStartSLOduration=7.728152211 podStartE2EDuration="21.782819745s" totalImagesPullingTime="14.054667534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:48.782542599 +0000 UTC m=+21.685739274" watchObservedRunningTime="2026-07-29 12:25:48.782819745 +0000 UTC m=+21.686016419" Jul 29 12:25:48.802745 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.800389 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x4qbv" podStartSLOduration=3.657947976 podStartE2EDuration="21.800372101s" totalImagesPullingTime="18.142424125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:48.79978387 +0000 UTC m=+21.702980534" watchObservedRunningTime="2026-07-29 12:25:48.800372101 +0000 UTC m=+21.703568778" Jul 29 12:25:48.817278 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.817225 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9svn6" podStartSLOduration=3.867296479 podStartE2EDuration="21.817205024s" totalImagesPullingTime="17.949908545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:48.816413357 +0000 UTC m=+21.719610033" watchObservedRunningTime="2026-07-29 12:25:48.817205024 +0000 UTC m=+21.720401702" Jul 29 12:25:48.855998 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:48.855966 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xm56m" podStartSLOduration=3.905676703 podStartE2EDuration="21.8559544s" totalImagesPullingTime="17.950277697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:48.855938082 +0000 UTC m=+21.759134757" watchObservedRunningTime="2026-07-29 12:25:48.8559544 +0000 UTC m=+21.759151073" Jul 29 12:25:49.652565 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.652540 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:49.652679 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.652548 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:49.652745 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:49.652699 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:49.652745 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:49.652733 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:49.715167 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.715135 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 12:25:49.761970 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.761937 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" event={"ID":"77e9deb9-c878-4140-b92a-c1ffeaa0abff","Type":"ContainerStarted","Data":"8ca5b6a55c655a5e522506e2418d31c852fba1d1f7fb9f2ad306889a0526d5d0"} Jul 29 12:25:49.765339 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.765310 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"7f9d06704a9590c93bbd5fa2aaa1fa1b3920d058448715d98523aab32c40b916"} Jul 29 12:25:49.765514 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.765492 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"2a2a6991a192e958c3a5dc7b12adc0335aa4f4b67847efa55556dc37f5da8c97"} Jul 29 12:25:49.765597 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.765556 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"09b6b52c3c60a5a31e2838e16a9f5f02b42e29654a09938ab0fb15fa245f4a47"} Jul 29 12:25:49.767083 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:49.767050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jxcrs" event={"ID":"61c35865-9c6a-439e-9a01-591169e965c6","Type":"ContainerStarted","Data":"5c061eb20fbe8f367c3a5f285bca43d7d4ccf35d6ff559950dc6f4a339e05a8c"} Jul 29 12:25:50.338284 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.338200 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:50.338850 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.338833 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:50.358365 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.356942 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jxcrs" podStartSLOduration=5.411322638 podStartE2EDuration="23.356922344s" totalImagesPullingTime="17.945599706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:49.786232009 +0000 UTC m=+22.689428710" watchObservedRunningTime="2026-07-29 12:25:50.356922344 +0000 UTC m=+23.260119025" Jul 29 12:25:50.529826 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.529721 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T12:25:49.71515192Z","UUID":"c12024d7-1413-423b-a20b-49eed0ecc45b","Handler":null,"Name":"","Endpoint":""} Jul 29 12:25:50.531326 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.531306 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 12:25:50.531438 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.531345 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 12:25:50.653275 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.653213 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:50.653572 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:50.653324 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:50.770390 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.770337 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:50.771151 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:50.771132 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cxphl" Jul 29 12:25:51.652594 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:51.652531 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:51.652594 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:51.652546 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:51.652894 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:51.652635 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:51.652894 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:51.652733 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:51.774337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:51.774302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" event={"ID":"77e9deb9-c878-4140-b92a-c1ffeaa0abff","Type":"ContainerStarted","Data":"2b962e7f2bf768d42a52f17f8f8df8bb4f55336a92fc65aa050ba371f5a6f26d"} Jul 29 12:25:51.777860 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:51.777822 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"1418d09971c8e0363e1610e7a39101d946d837712da0cfb1a018495b3fecbd8c"} Jul 29 12:25:51.795094 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:51.795056 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhpd7" podStartSLOduration=4.850199948 podStartE2EDuration="24.795036962s" totalImagesPullingTime="19.944837014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:51.794942274 +0000 UTC m=+24.698138950" watchObservedRunningTime="2026-07-29 12:25:51.795036962 +0000 UTC m=+24.698233638" Jul 29 12:25:52.652357 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:52.652326 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:52.652593 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:52.652454 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:53.655624 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.655470 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:53.656283 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.655519 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:53.656283 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:53.655692 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:53.656283 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:53.655757 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:53.784701 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.784631 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" event={"ID":"bf5272ab-7183-4547-90c2-197987a1e3c1","Type":"ContainerStarted","Data":"e14e1d5ed218b5142820ab5bb54941165e5662d40b3dea0ebd95cb8c72e6411b"} Jul 29 12:25:53.784900 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.784874 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:53.786582 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.786557 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9942c8c-5762-439f-8ce1-8bd261960a9e" containerID="2ae5a9ebbbccf4614cb9a053c6aa52ead8c52fde4acc969ee6c296bee1b7d298" exitCode=0 Jul 29 12:25:53.786694 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.786595 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerDied","Data":"2ae5a9ebbbccf4614cb9a053c6aa52ead8c52fde4acc969ee6c296bee1b7d298"} Jul 29 12:25:53.801195 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.801176 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:53.812518 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:53.812479 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" podStartSLOduration=8.837547866 podStartE2EDuration="26.812449268s" totalImagesPullingTime="17.974901402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:25:53.810020724 +0000 UTC m=+26.713217399" watchObservedRunningTime="2026-07-29 12:25:53.812449268 +0000 UTC m=+26.715646008" Jul 29 12:25:54.652587 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:54.652338 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:54.652771 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:54.652668 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:54.788729 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:54.788700 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:54.789164 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:54.788766 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:54.802480 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:54.802445 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:25:55.238328 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.238294 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:55.332892 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.332869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:55.613902 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.613872 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lpk5s"] Jul 29 12:25:55.614045 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.614031 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:55.614167 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:55.614140 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:55.617169 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.616960 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7m5ln"] Jul 29 12:25:55.617169 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.617044 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:55.617169 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:55.617111 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:55.617649 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.617629 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zcf8c"] Jul 29 12:25:55.617738 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.617726 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:55.617839 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:55.617821 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:55.791910 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.791886 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9942c8c-5762-439f-8ce1-8bd261960a9e" containerID="1563f432408c22b05e2a89f751a40c3435f2e711f0864b02883fece9d7cc6d94" exitCode=0 Jul 29 12:25:55.792281 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:55.791963 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerDied","Data":"1563f432408c22b05e2a89f751a40c3435f2e711f0864b02883fece9d7cc6d94"} Jul 29 12:25:56.412176 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:56.412152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:56.512354 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:56.512331 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:57.576151 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.576126 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:57.655280 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.655169 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:57.655369 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.655210 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:57.655369 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:57.655350 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:57.655487 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.655220 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:57.655487 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:57.655470 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:25:57.655591 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:57.655560 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:57.713659 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.713642 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:57.798227 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.798203 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9942c8c-5762-439f-8ce1-8bd261960a9e" containerID="7d652d54d500c5446ec0cd6492ac1d5a370c4b6de36bdff2ca5f8a393b83ba96" exitCode=0 Jul 29 12:25:57.798311 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.798265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerDied","Data":"7d652d54d500c5446ec0cd6492ac1d5a370c4b6de36bdff2ca5f8a393b83ba96"} Jul 29 12:25:57.803974 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:57.803957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:25:59.652359 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:59.652331 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:25:59.652359 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:59.652349 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:25:59.652938 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:59.652453 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7m5ln" podUID="9562d83d-4984-4e18-be4c-c37408d9169c" Jul 29 12:25:59.652938 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:59.652534 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zcf8c" podUID="7d7e341b-4606-4c63-a278-9f9d71a68296" Jul 29 12:25:59.652938 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:25:59.652580 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:25:59.652938 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:25:59.652638 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lpk5s" podUID="a1972259-e332-4fbb-b3db-247f6bdb3f95" Jul 29 12:26:00.215959 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:00.215922 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:00.288179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:00.288148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:01.108862 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.108821 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:01.209738 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.209674 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:01.220499 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.220470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:26:01.220632 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.220581 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 12:26:01.220632 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.220601 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 12:26:01.220632 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.220613 2563 projected.go:196] Error preparing data for projected volume kube-api-access-qxxqg for pod openshift-network-diagnostics/network-check-target-7m5ln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:26:01.220777 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.220670 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg podName:9562d83d-4984-4e18-be4c-c37408d9169c nodeName:}" failed. No retries permitted until 2026-07-29 12:26:33.220656121 +0000 UTC m=+66.123852783 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qxxqg" (UniqueName: "kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg") pod "network-check-target-7m5ln" (UID: "9562d83d-4984-4e18-be4c-c37408d9169c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 12:26:01.321128 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.321092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:26:01.321291 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.321196 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:26:01.321291 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.321247 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:26:01.321396 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.321293 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 12:26:01.321396 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.321312 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs podName:7d7e341b-4606-4c63-a278-9f9d71a68296 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:33.321297104 +0000 UTC m=+66.224493758 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs") pod "network-metrics-daemon-zcf8c" (UID: "7d7e341b-4606-4c63-a278-9f9d71a68296") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 12:26:01.321396 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.321332 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret podName:a1972259-e332-4fbb-b3db-247f6bdb3f95 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:33.321322326 +0000 UTC m=+66.224518980 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret") pod "global-pull-secret-syncer-lpk5s" (UID: "a1972259-e332-4fbb-b3db-247f6bdb3f95") : object "kube-system"/"original-pull-secret" not registered Jul 29 12:26:01.421734 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.421700 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-159.ec2.internal" event="NodeReady" Jul 29 12:26:01.421883 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.421848 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 12:26:01.475474 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.475387 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4kxgm"] Jul 29 12:26:01.477808 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.477785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.478903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.478883 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cj9cn"] Jul 29 12:26:01.480713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.480479 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 12:26:01.480713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.480496 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 12:26:01.480713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.480529 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 12:26:01.480713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.480530 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2fqhh\"" Jul 29 12:26:01.480954 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.480937 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.482963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.482944 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 12:26:01.483249 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.483235 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 12:26:01.483338 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.483296 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zrh5t\"" Jul 29 12:26:01.489999 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.489981 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4kxgm"] Jul 29 12:26:01.493059 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.493040 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cj9cn"] Jul 29 12:26:01.623060 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.623040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d69591be-c441-4afe-8c26-97b512243e07-cert\") pod \"ingress-canary-4kxgm\" (UID: \"d69591be-c441-4afe-8c26-97b512243e07\") " pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.623135 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.623071 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bnbb\" (UniqueName: \"kubernetes.io/projected/d69591be-c441-4afe-8c26-97b512243e07-kube-api-access-9bnbb\") pod \"ingress-canary-4kxgm\" (UID: \"d69591be-c441-4afe-8c26-97b512243e07\") " pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.623135 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.623094 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34dabae6-f013-4498-9c47-de54f756de64-config-volume\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.623213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.623142 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.623213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.623179 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34dabae6-f013-4498-9c47-de54f756de64-tmp-dir\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.623277 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.623227 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdpj5\" (UniqueName: \"kubernetes.io/projected/34dabae6-f013-4498-9c47-de54f756de64-kube-api-access-bdpj5\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.652302 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.652281 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:26:01.652382 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.652281 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:26:01.652449 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.652423 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:26:01.655351 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.655334 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p4f75\"" Jul 29 12:26:01.655612 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.655574 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 12:26:01.655711 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.655663 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hdxp9\"" Jul 29 12:26:01.655796 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.655780 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 12:26:01.655872 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.655857 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 12:26:01.656087 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.656073 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 12:26:01.724098 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.724072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.724211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.724112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34dabae6-f013-4498-9c47-de54f756de64-tmp-dir\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.724211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.724133 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdpj5\" (UniqueName: \"kubernetes.io/projected/34dabae6-f013-4498-9c47-de54f756de64-kube-api-access-bdpj5\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.724211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.724157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d69591be-c441-4afe-8c26-97b512243e07-cert\") pod \"ingress-canary-4kxgm\" (UID: \"d69591be-c441-4afe-8c26-97b512243e07\") " pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.724211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.724178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bnbb\" (UniqueName: \"kubernetes.io/projected/d69591be-c441-4afe-8c26-97b512243e07-kube-api-access-9bnbb\") pod \"ingress-canary-4kxgm\" (UID: \"d69591be-c441-4afe-8c26-97b512243e07\") " pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.724211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.724204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34dabae6-f013-4498-9c47-de54f756de64-config-volume\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.725239 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.724766 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 12:26:01.725239 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.725031 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34dabae6-f013-4498-9c47-de54f756de64-tmp-dir\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.725410 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:01.725343 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls podName:34dabae6-f013-4498-9c47-de54f756de64 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:02.225303362 +0000 UTC m=+35.128500040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls") pod "dns-default-cj9cn" (UID: "34dabae6-f013-4498-9c47-de54f756de64") : secret "dns-default-metrics-tls" not found Jul 29 12:26:01.726306 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.725629 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34dabae6-f013-4498-9c47-de54f756de64-config-volume\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.731545 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.731525 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d69591be-c441-4afe-8c26-97b512243e07-cert\") pod \"ingress-canary-4kxgm\" (UID: \"d69591be-c441-4afe-8c26-97b512243e07\") " pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.734385 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.734362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdpj5\" (UniqueName: \"kubernetes.io/projected/34dabae6-f013-4498-9c47-de54f756de64-kube-api-access-bdpj5\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:01.735128 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.735104 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bnbb\" (UniqueName: \"kubernetes.io/projected/d69591be-c441-4afe-8c26-97b512243e07-kube-api-access-9bnbb\") pod \"ingress-canary-4kxgm\" (UID: \"d69591be-c441-4afe-8c26-97b512243e07\") " pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.789447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.789420 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4kxgm" Jul 29 12:26:01.809698 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.809663 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9942c8c-5762-439f-8ce1-8bd261960a9e" containerID="cef4fec86c88a2352999aeaccf062cb39d9fa82df3367a97c70e0f2fc7e76a12" exitCode=0 Jul 29 12:26:01.809793 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.809710 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerDied","Data":"cef4fec86c88a2352999aeaccf062cb39d9fa82df3367a97c70e0f2fc7e76a12"} Jul 29 12:26:01.924931 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:01.924906 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4kxgm"] Jul 29 12:26:01.928481 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:01.928437 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69591be_c441_4afe_8c26_97b512243e07.slice/crio-13f7dffa087150fbb338c259170329990287d4ae5acb2ce9b55943374fe85394 WatchSource:0}: Error finding container 13f7dffa087150fbb338c259170329990287d4ae5acb2ce9b55943374fe85394: Status 404 returned error can't find the container with id 13f7dffa087150fbb338c259170329990287d4ae5acb2ce9b55943374fe85394 Jul 29 12:26:02.228444 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:02.228418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:02.228937 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:02.228537 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 12:26:02.228937 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:02.228586 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls podName:34dabae6-f013-4498-9c47-de54f756de64 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:03.228570186 +0000 UTC m=+36.131766860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls") pod "dns-default-cj9cn" (UID: "34dabae6-f013-4498-9c47-de54f756de64") : secret "dns-default-metrics-tls" not found Jul 29 12:26:02.815056 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:02.815027 2563 generic.go:356] "Generic (PLEG): container finished" podID="c9942c8c-5762-439f-8ce1-8bd261960a9e" containerID="207e9242cd724404f6c805b65079a6cac6f616d5244b7a3b43729954a85463ca" exitCode=0 Jul 29 12:26:02.815216 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:02.815112 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerDied","Data":"207e9242cd724404f6c805b65079a6cac6f616d5244b7a3b43729954a85463ca"} Jul 29 12:26:02.816272 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:02.816115 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4kxgm" event={"ID":"d69591be-c441-4afe-8c26-97b512243e07","Type":"ContainerStarted","Data":"13f7dffa087150fbb338c259170329990287d4ae5acb2ce9b55943374fe85394"} Jul 29 12:26:03.037514 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:03.037475 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:03.234909 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:03.234883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:03.235297 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:03.235045 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 12:26:03.235297 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:03.235113 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls podName:34dabae6-f013-4498-9c47-de54f756de64 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.235093955 +0000 UTC m=+38.138290613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls") pod "dns-default-cj9cn" (UID: "34dabae6-f013-4498-9c47-de54f756de64") : secret "dns-default-metrics-tls" not found Jul 29 12:26:03.817217 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:03.817182 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:03.822142 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:03.822115 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" event={"ID":"c9942c8c-5762-439f-8ce1-8bd261960a9e","Type":"ContainerStarted","Data":"7bfc5558967086fb473304b827e417139fb3ade60c29d3f33dee4bb18638dcba"} Jul 29 12:26:03.847545 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:03.847497 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6vqb2" podStartSLOduration=7.6143963150000005 podStartE2EDuration="36.847478179s" totalImagesPullingTime="29.233081864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:03.845864639 +0000 UTC m=+36.749061317" watchObservedRunningTime="2026-07-29 12:26:03.847478179 +0000 UTC m=+36.750674850" Jul 29 12:26:03.918160 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:03.918133 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:04.045672 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.045638 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:04.051400 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.051374 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cmgdg"] Jul 29 12:26:04.053538 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.053519 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.056949 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.056925 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 12:26:04.057151 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.057134 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 12:26:04.057443 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.057421 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 12:26:04.057443 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.057439 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 12:26:04.057822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.057806 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lp4bb\"" Jul 29 12:26:04.071158 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.071096 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmgdg"] Jul 29 12:26:04.142880 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.142844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.143018 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.142922 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vrgd\" (UniqueName: \"kubernetes.io/projected/7b259a3c-d681-4dab-a491-38acf064c970-kube-api-access-8vrgd\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.143018 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.142968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b259a3c-d681-4dab-a491-38acf064c970-data-volume\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.143018 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.142994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.143191 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.143041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b259a3c-d681-4dab-a491-38acf064c970-crio-socket\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.147958 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.147929 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:04.244312 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vrgd\" (UniqueName: \"kubernetes.io/projected/7b259a3c-d681-4dab-a491-38acf064c970-kube-api-access-8vrgd\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244335 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b259a3c-d681-4dab-a491-38acf064c970-data-volume\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244353 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b259a3c-d681-4dab-a491-38acf064c970-crio-socket\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.244541 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.244604 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:04.744586241 +0000 UTC m=+37.647782916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : secret "insights-runtime-extractor-tls" not found Jul 29 12:26:04.244689 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.244541 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 12:26:04.245000 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244727 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7b259a3c-d681-4dab-a491-38acf064c970-crio-socket\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.245000 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.244734 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:04.744714086 +0000 UTC m=+37.647910760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : configmap "kube-rbac-proxy" not found Jul 29 12:26:04.245088 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.244911 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b259a3c-d681-4dab-a491-38acf064c970-data-volume\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.255940 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.255916 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vrgd\" (UniqueName: \"kubernetes.io/projected/7b259a3c-d681-4dab-a491-38acf064c970-kube-api-access-8vrgd\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.314393 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.314367 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:04.579163 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.579132 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:04.748447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.748416 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.748624 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.748501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:04.748624 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.748576 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 12:26:04.748726 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.748647 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.748626384 +0000 UTC m=+38.651823053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : configmap "kube-rbac-proxy" not found Jul 29 12:26:04.748726 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.748696 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 12:26:04.748833 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:04.748740 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:05.748729945 +0000 UTC m=+38.651926600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : secret "insights-runtime-extractor-tls" not found Jul 29 12:26:04.806414 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.806381 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:04.927277 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:04.927240 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:05.253295 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.253269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:05.253953 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:05.253387 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 12:26:05.253953 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:05.253438 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls podName:34dabae6-f013-4498-9c47-de54f756de64 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:09.253422465 +0000 UTC m=+42.156619120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls") pod "dns-default-cj9cn" (UID: "34dabae6-f013-4498-9c47-de54f756de64") : secret "dns-default-metrics-tls" not found Jul 29 12:26:05.466103 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.466069 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:05.565623 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.565561 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:05.682422 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.682396 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv"] Jul 29 12:26:05.712957 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.712932 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv"] Jul 29 12:26:05.713056 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.713021 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" Jul 29 12:26:05.715608 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.715585 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t5sl8\"" Jul 29 12:26:05.716389 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.716372 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 29 12:26:05.716502 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.716398 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 29 12:26:05.729992 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.729972 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:05.758012 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.757991 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:05.758081 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.758047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:05.758124 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:05.758104 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 12:26:05.758124 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:05.758122 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 12:26:05.758192 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:05.758153 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:07.758141393 +0000 UTC m=+40.661338047 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : configmap "kube-rbac-proxy" not found Jul 29 12:26:05.758192 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:05.758166 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:07.758160155 +0000 UTC m=+40.661356809 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : secret "insights-runtime-extractor-tls" not found Jul 29 12:26:05.827823 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.827759 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4kxgm" event={"ID":"d69591be-c441-4afe-8c26-97b512243e07","Type":"ContainerStarted","Data":"d8a63c68f01cfbae2d568ac9894d2a497aeb47ba1e4b70d3d77395bcf15c279b"} Jul 29 12:26:05.827919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.827882 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:05.843967 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.843928 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4kxgm" podStartSLOduration=1.968743957 podStartE2EDuration="4.84391463s" totalImagesPullingTime="2.875170673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:05.843485891 +0000 UTC m=+38.746682566" watchObservedRunningTime="2026-07-29 12:26:05.84391463 +0000 UTC m=+38.747111305" Jul 29 12:26:05.858683 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.858659 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5psf\" (UniqueName: \"kubernetes.io/projected/44d62cfa-a814-4009-8e9e-002507edae0a-kube-api-access-q5psf\") pod \"migrator-6865c9967-kf9vv\" (UID: \"44d62cfa-a814-4009-8e9e-002507edae0a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" Jul 29 12:26:05.959777 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.959747 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5psf\" (UniqueName: \"kubernetes.io/projected/44d62cfa-a814-4009-8e9e-002507edae0a-kube-api-access-q5psf\") pod \"migrator-6865c9967-kf9vv\" (UID: \"44d62cfa-a814-4009-8e9e-002507edae0a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" Jul 29 12:26:05.968903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:05.968881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5psf\" (UniqueName: \"kubernetes.io/projected/44d62cfa-a814-4009-8e9e-002507edae0a-kube-api-access-q5psf\") pod \"migrator-6865c9967-kf9vv\" (UID: \"44d62cfa-a814-4009-8e9e-002507edae0a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" Jul 29 12:26:06.021879 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.021858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" Jul 29 12:26:06.108068 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.108037 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lk8nj"] Jul 29 12:26:06.138654 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.138627 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lk8nj"] Jul 29 12:26:06.138754 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.138738 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.141633 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.141594 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 29 12:26:06.141755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.141647 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 29 12:26:06.141755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.141659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:06.141755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.141669 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5ppjc\"" Jul 29 12:26:06.141755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.141594 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 29 12:26:06.141933 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.141594 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 29 12:26:06.170319 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.170301 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv"] Jul 29 12:26:06.173257 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:06.173233 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d62cfa_a814_4009_8e9e_002507edae0a.slice/crio-7ef6a3c2aead9dd6cb8623befb44de54ed8de46702764b551185a769d35d051a WatchSource:0}: Error finding container 7ef6a3c2aead9dd6cb8623befb44de54ed8de46702764b551185a769d35d051a: Status 404 returned error can't find the container with id 7ef6a3c2aead9dd6cb8623befb44de54ed8de46702764b551185a769d35d051a Jul 29 12:26:06.209844 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.209826 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xm56m_700bc436-2f38-40e7-a91d-ac67dad37ed0/dns-node-resolver/0.log" Jul 29 12:26:06.243736 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.243714 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:06.262793 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.262774 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-signing-cabundle\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.263028 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.262895 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-signing-key\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.263028 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.262930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z77km\" (UniqueName: \"kubernetes.io/projected/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-kube-api-access-z77km\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.364205 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.364157 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-signing-key\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.364205 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.364184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z77km\" (UniqueName: \"kubernetes.io/projected/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-kube-api-access-z77km\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.364205 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.364203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-signing-cabundle\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.364862 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.364846 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-signing-cabundle\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.366400 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.366380 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-signing-key\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.372624 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.372602 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z77km\" (UniqueName: \"kubernetes.io/projected/2e627cd6-ed3b-4e64-a04e-2e8ddf268674-kube-api-access-z77km\") pod \"service-ca-658d7b7c78-lk8nj\" (UID: \"2e627cd6-ed3b-4e64-a04e-2e8ddf268674\") " pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.447665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.447647 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" Jul 29 12:26:06.602202 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.602174 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-lk8nj"] Jul 29 12:26:06.604652 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:06.604632 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e627cd6_ed3b_4e64_a04e_2e8ddf268674.slice/crio-22ea20532e4e6b3705bd7c291945dc36e5847cada5d20147ab655fe549e90756 WatchSource:0}: Error finding container 22ea20532e4e6b3705bd7c291945dc36e5847cada5d20147ab655fe549e90756: Status 404 returned error can't find the container with id 22ea20532e4e6b3705bd7c291945dc36e5847cada5d20147ab655fe549e90756 Jul 29 12:26:06.814918 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.814894 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9svn6_7e324f8b-44ff-4145-9926-4078115e22cb/node-ca/0.log" Jul 29 12:26:06.830507 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.830474 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" event={"ID":"2e627cd6-ed3b-4e64-a04e-2e8ddf268674","Type":"ContainerStarted","Data":"22ea20532e4e6b3705bd7c291945dc36e5847cada5d20147ab655fe549e90756"} Jul 29 12:26:06.831505 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:06.831482 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" event={"ID":"44d62cfa-a814-4009-8e9e-002507edae0a","Type":"ContainerStarted","Data":"7ef6a3c2aead9dd6cb8623befb44de54ed8de46702764b551185a769d35d051a"} Jul 29 12:26:07.113864 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:07.113785 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:07.614868 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:07.614834 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4kxgm_d69591be-c441-4afe-8c26-97b512243e07/serve-healthcheck-canary/0.log" Jul 29 12:26:07.618833 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:07.618795 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:07.773871 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:07.773843 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:07.774000 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:07.773924 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:07.774056 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:07.774027 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 12:26:07.774102 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:07.774091 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:11.774072239 +0000 UTC m=+44.677268909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : secret "insights-runtime-extractor-tls" not found Jul 29 12:26:07.774300 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:07.774281 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7b259a3c-d681-4dab-a491-38acf064c970-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:08.838026 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:08.838000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" event={"ID":"44d62cfa-a814-4009-8e9e-002507edae0a","Type":"ContainerStarted","Data":"989b835539f844fbc6335bb2db47663829a4e709e67c7d37ef759fc91cc9478c"} Jul 29 12:26:08.838392 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:08.838051 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" event={"ID":"44d62cfa-a814-4009-8e9e-002507edae0a","Type":"ContainerStarted","Data":"c4f8c09da426b97389e75a1e839685c439329bedf8fd1cad782c81899a8fa89a"} Jul 29 12:26:08.861271 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:08.861230 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-kf9vv" podStartSLOduration=1.992405451 podStartE2EDuration="3.8612179s" totalImagesPullingTime="1.868812449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:08.86107702 +0000 UTC m=+41.764273696" watchObservedRunningTime="2026-07-29 12:26:08.8612179 +0000 UTC m=+41.764414575" Jul 29 12:26:09.284834 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:09.284805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:09.284950 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:09.284906 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 12:26:09.284950 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:09.284947 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls podName:34dabae6-f013-4498-9c47-de54f756de64 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:17.284935858 +0000 UTC m=+50.188132513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls") pod "dns-default-cj9cn" (UID: "34dabae6-f013-4498-9c47-de54f756de64") : secret "dns-default-metrics-tls" not found Jul 29 12:26:09.841443 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:09.841413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" event={"ID":"2e627cd6-ed3b-4e64-a04e-2e8ddf268674","Type":"ContainerStarted","Data":"376fa06ae17b03c024b1abc7a28b71715aa2679c90c1fec80bff3b71b6f9c1e6"} Jul 29 12:26:09.861546 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:09.861501 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-lk8nj" podStartSLOduration=1.4276458029999999 podStartE2EDuration="3.86148678s" totalImagesPullingTime="2.433840977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:09.86052378 +0000 UTC m=+42.763720456" watchObservedRunningTime="2026-07-29 12:26:09.86148678 +0000 UTC m=+42.764683455" Jul 29 12:26:10.570855 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:10.570821 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:11.052681 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:11.052652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:11.282814 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:11.282785 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:11.446308 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:11.446238 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:11.543204 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:11.543179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:11.802526 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:11.802498 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:11.802662 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:11.802644 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 12:26:11.802719 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:11.802709 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls podName:7b259a3c-d681-4dab-a491-38acf064c970 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:19.802695303 +0000 UTC m=+52.705891956 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmgdg" (UID: "7b259a3c-d681-4dab-a491-38acf064c970") : secret "insights-runtime-extractor-tls" not found Jul 29 12:26:12.350392 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:12.350361 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:12.747515 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:12.747489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:12.933828 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:12.933798 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:13.142784 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:13.142705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:13.455284 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:13.455212 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:14.276406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:14.276374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:14.380502 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:14.380455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:14.655935 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:14.655853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:14.756774 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:14.756743 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:14.937965 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:14.937899 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:15.137243 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:15.137217 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:15.238924 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:15.238901 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:15.346706 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:15.346688 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:15.420919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:15.420893 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:16.162936 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:16.162894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:16.739830 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:16.739801 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:16.871903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:16.871878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:16.969853 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:16.969831 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:17.300999 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.300971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:17.342414 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.342389 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:17.344641 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.344618 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34dabae6-f013-4498-9c47-de54f756de64-metrics-tls\") pod \"dns-default-cj9cn\" (UID: \"34dabae6-f013-4498-9c47-de54f756de64\") " pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:17.396678 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.396651 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:17.529722 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.528883 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cj9cn"] Jul 29 12:26:17.533651 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:17.533615 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34dabae6_f013_4498_9c47_de54f756de64.slice/crio-e6c8984bb8d4035c299e22c0319904146d2ccbc01d29aa7cd13ed502c0e0820e WatchSource:0}: Error finding container e6c8984bb8d4035c299e22c0319904146d2ccbc01d29aa7cd13ed502c0e0820e: Status 404 returned error can't find the container with id e6c8984bb8d4035c299e22c0319904146d2ccbc01d29aa7cd13ed502c0e0820e Jul 29 12:26:17.543043 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.543021 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:17.861029 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.861002 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cj9cn" event={"ID":"34dabae6-f013-4498-9c47-de54f756de64","Type":"ContainerStarted","Data":"e6c8984bb8d4035c299e22c0319904146d2ccbc01d29aa7cd13ed502c0e0820e"} Jul 29 12:26:17.944675 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:17.944648 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:18.543311 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:18.543275 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:18.817381 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:18.817277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:18.940881 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:18.940853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:19.351811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:19.351773 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:19.547832 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:19.547798 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:19.861700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:19.861671 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:19.863769 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:19.863748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7b259a3c-d681-4dab-a491-38acf064c970-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmgdg\" (UID: \"7b259a3c-d681-4dab-a491-38acf064c970\") " pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:19.942680 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:19.942655 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:19.965959 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:19.965940 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmgdg" Jul 29 12:26:20.201221 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.201197 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmgdg"] Jul 29 12:26:20.203902 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:20.203876 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b259a3c_d681_4dab_a491_38acf064c970.slice/crio-07c9ed75cdd1343c02e415dceccee1ebd008c55700d101011b8a60fc2d5a6e7d WatchSource:0}: Error finding container 07c9ed75cdd1343c02e415dceccee1ebd008c55700d101011b8a60fc2d5a6e7d: Status 404 returned error can't find the container with id 07c9ed75cdd1343c02e415dceccee1ebd008c55700d101011b8a60fc2d5a6e7d Jul 29 12:26:20.535283 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.535257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:20.874616 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.874573 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cj9cn" event={"ID":"34dabae6-f013-4498-9c47-de54f756de64","Type":"ContainerStarted","Data":"0a0a8d343c159e1a933a497fba4c11855ec892b48a0108695769e92fdacb9397"} Jul 29 12:26:20.875084 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.874677 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cj9cn" event={"ID":"34dabae6-f013-4498-9c47-de54f756de64","Type":"ContainerStarted","Data":"2ee7ebbd45ce4d2eec1b42ed4431be8748fa81cd4f3b9f068dcda5653f3fa9c6"} Jul 29 12:26:20.875084 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.874953 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:20.877570 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.877513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmgdg" event={"ID":"7b259a3c-d681-4dab-a491-38acf064c970","Type":"ContainerStarted","Data":"07c9ed75cdd1343c02e415dceccee1ebd008c55700d101011b8a60fc2d5a6e7d"} Jul 29 12:26:20.894877 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.894834 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cj9cn" podStartSLOduration=17.345914662 podStartE2EDuration="19.894817685s" totalImagesPullingTime="2.548903023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:20.894310008 +0000 UTC m=+53.797506685" watchObservedRunningTime="2026-07-29 12:26:20.894817685 +0000 UTC m=+53.798014360" Jul 29 12:26:20.944756 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:20.944699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:21.154602 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:21.154527 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:21.333932 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:21.333897 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:25.732715 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:25.732617 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:25.832282 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:25.832245 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:25.898346 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:25.898313 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmgdg" event={"ID":"7b259a3c-d681-4dab-a491-38acf064c970","Type":"ContainerStarted","Data":"78bc550e21875f1d02ea361cb45301c588c218533f72c2265e2a5cd6b9740165"} Jul 29 12:26:25.907863 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:25.907835 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.056940 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.056882 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.383697 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.383613 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.484777 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.484747 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.584716 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.584690 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.778321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.778287 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.815985 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.815962 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xvhcg" Jul 29 12:26:26.854744 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.854712 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:26.987565 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:26.987533 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:27.089218 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:27.089141 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:27.200695 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:27.200665 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:27.399447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:27.399384 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:27.762798 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:27.762767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:28.421547 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.421523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:28.521168 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.521145 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:28.910186 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.910076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmgdg" event={"ID":"7b259a3c-d681-4dab-a491-38acf064c970","Type":"ContainerStarted","Data":"863afa179b1491359f8ce2b27ca614c20b4ab726333ca8628d26cda5150342cc"} Jul 29 12:26:28.994684 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.994650 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh"] Jul 29 12:26:28.997015 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.996991 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6"] Jul 29 12:26:28.997176 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.997159 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:28.998850 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:28.998830 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.000557 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.000531 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 29 12:26:29.000696 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.000668 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 29 12:26:29.001384 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.001364 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9wgjv\"" Jul 29 12:26:29.001605 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.001588 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 29 12:26:29.001735 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.001670 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 29 12:26:29.005387 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.005324 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 29 12:26:29.005537 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.005490 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 29 12:26:29.005635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.005624 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 29 12:26:29.006262 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.006244 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 29 12:26:29.021587 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.021568 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh"] Jul 29 12:26:29.022557 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.022519 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6"] Jul 29 12:26:29.033509 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.033614 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033532 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.033614 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033557 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/58aa9529-5196-4b4c-98de-3b7469d0a375-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-659786d874-vbfj6\" (UID: \"58aa9529-5196-4b4c-98de-3b7469d0a375\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.033614 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-hub\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.033773 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033627 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/09aa5fa8-e30b-432d-b07c-5c71d21f241b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.033773 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033676 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chpn6\" (UniqueName: \"kubernetes.io/projected/09aa5fa8-e30b-432d-b07c-5c71d21f241b-kube-api-access-chpn6\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.033773 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033698 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrf9\" (UniqueName: \"kubernetes.io/projected/58aa9529-5196-4b4c-98de-3b7469d0a375-kube-api-access-xgrf9\") pod \"managed-serviceaccount-addon-agent-659786d874-vbfj6\" (UID: \"58aa9529-5196-4b4c-98de-3b7469d0a375\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.033773 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.033755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-ca\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.057491 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.057449 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:29.134196 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-hub\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.134317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/09aa5fa8-e30b-432d-b07c-5c71d21f241b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.134317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chpn6\" (UniqueName: \"kubernetes.io/projected/09aa5fa8-e30b-432d-b07c-5c71d21f241b-kube-api-access-chpn6\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.134317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134286 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrf9\" (UniqueName: \"kubernetes.io/projected/58aa9529-5196-4b4c-98de-3b7469d0a375-kube-api-access-xgrf9\") pod \"managed-serviceaccount-addon-agent-659786d874-vbfj6\" (UID: \"58aa9529-5196-4b4c-98de-3b7469d0a375\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.134511 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134333 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-ca\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.134511 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134379 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.134511 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134417 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.134511 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.134451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/58aa9529-5196-4b4c-98de-3b7469d0a375-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-659786d874-vbfj6\" (UID: \"58aa9529-5196-4b4c-98de-3b7469d0a375\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.135112 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.135085 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/09aa5fa8-e30b-432d-b07c-5c71d21f241b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.137488 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.137205 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-hub\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.137488 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.137329 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/58aa9529-5196-4b4c-98de-3b7469d0a375-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-659786d874-vbfj6\" (UID: \"58aa9529-5196-4b4c-98de-3b7469d0a375\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.137488 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.137402 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.137645 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.137527 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-ca\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.137767 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.137741 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09aa5fa8-e30b-432d-b07c-5c71d21f241b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.152598 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.152574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrf9\" (UniqueName: \"kubernetes.io/projected/58aa9529-5196-4b4c-98de-3b7469d0a375-kube-api-access-xgrf9\") pod \"managed-serviceaccount-addon-agent-659786d874-vbfj6\" (UID: \"58aa9529-5196-4b4c-98de-3b7469d0a375\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.155196 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.155177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chpn6\" (UniqueName: \"kubernetes.io/projected/09aa5fa8-e30b-432d-b07c-5c71d21f241b-kube-api-access-chpn6\") pod \"cluster-proxy-proxy-agent-6889447bf9-fxthh\" (UID: \"09aa5fa8-e30b-432d-b07c-5c71d21f241b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.161005 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.160956 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:29.279947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.279900 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:29.321669 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.321643 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" Jul 29 12:26:29.328969 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.328949 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" Jul 29 12:26:29.402451 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.402387 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:29.465526 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.465452 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6"] Jul 29 12:26:29.469002 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:29.468971 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58aa9529_5196_4b4c_98de_3b7469d0a375.slice/crio-0fdb202312a4701a0c3a85e9e58c089840efafb18dc81f196f78d35d59b2cd82 WatchSource:0}: Error finding container 0fdb202312a4701a0c3a85e9e58c089840efafb18dc81f196f78d35d59b2cd82: Status 404 returned error can't find the container with id 0fdb202312a4701a0c3a85e9e58c089840efafb18dc81f196f78d35d59b2cd82 Jul 29 12:26:29.488227 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.488184 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:29.493771 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.493700 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh"] Jul 29 12:26:29.496059 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:29.496034 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09aa5fa8_e30b_432d_b07c_5c71d21f241b.slice/crio-cc4e99af29dd4ea1b5b682ec646c7a2109ca665f661675bfa68f696446820c39 WatchSource:0}: Error finding container cc4e99af29dd4ea1b5b682ec646c7a2109ca665f661675bfa68f696446820c39: Status 404 returned error can't find the container with id cc4e99af29dd4ea1b5b682ec646c7a2109ca665f661675bfa68f696446820c39 Jul 29 12:26:29.914361 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.914328 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" event={"ID":"58aa9529-5196-4b4c-98de-3b7469d0a375","Type":"ContainerStarted","Data":"0fdb202312a4701a0c3a85e9e58c089840efafb18dc81f196f78d35d59b2cd82"} Jul 29 12:26:29.915788 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:29.915749 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" event={"ID":"09aa5fa8-e30b-432d-b07c-5c71d21f241b","Type":"ContainerStarted","Data":"cc4e99af29dd4ea1b5b682ec646c7a2109ca665f661675bfa68f696446820c39"} Jul 29 12:26:30.276790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:30.276722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:30.407158 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:30.407076 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:30.883549 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:30.883519 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cj9cn" Jul 29 12:26:33.179590 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.179559 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:33.270941 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.270910 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:26:33.274816 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.274792 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 12:26:33.282023 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.281975 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:33.284019 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.283994 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 12:26:33.293932 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.293908 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxxqg\" (UniqueName: \"kubernetes.io/projected/9562d83d-4984-4e18-be4c-c37408d9169c-kube-api-access-qxxqg\") pod \"network-check-target-7m5ln\" (UID: \"9562d83d-4984-4e18-be4c-c37408d9169c\") " pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:26:33.356134 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.356109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:33.372126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.372105 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:26:33.372227 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.372142 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:26:33.374925 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.374908 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 12:26:33.375821 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.375805 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 12:26:33.385187 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.385169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d7e341b-4606-4c63-a278-9f9d71a68296-metrics-certs\") pod \"network-metrics-daemon-zcf8c\" (UID: \"7d7e341b-4606-4c63-a278-9f9d71a68296\") " pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:26:33.385283 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.385268 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a1972259-e332-4fbb-b3db-247f6bdb3f95-original-pull-secret\") pod \"global-pull-secret-syncer-lpk5s\" (UID: \"a1972259-e332-4fbb-b3db-247f6bdb3f95\") " pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:26:33.477653 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.477625 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p4f75\"" Jul 29 12:26:33.485039 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.485016 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zcf8c" Jul 29 12:26:33.485454 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.485436 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hdxp9\"" Jul 29 12:26:33.487541 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.487522 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lpk5s" Jul 29 12:26:33.493665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.493648 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:26:33.833982 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.833903 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:33.941317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:33.941278 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:34.944419 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:34.944389 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:35.386635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.386611 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zcf8c"] Jul 29 12:26:35.390278 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:35.389964 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7e341b_4606_4c63_a278_9f9d71a68296.slice/crio-733a3e55bc5cddaabd113e40dcda2fde5454f3913fc067c71573e566d50a37ab WatchSource:0}: Error finding container 733a3e55bc5cddaabd113e40dcda2fde5454f3913fc067c71573e566d50a37ab: Status 404 returned error can't find the container with id 733a3e55bc5cddaabd113e40dcda2fde5454f3913fc067c71573e566d50a37ab Jul 29 12:26:35.398913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.398856 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7m5ln"] Jul 29 12:26:35.401868 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:35.401849 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9562d83d_4984_4e18_be4c_c37408d9169c.slice/crio-79fe7ed9a238c62401b1f71acf077bdcd644124e17266bf1b80c3a49debd89e0 WatchSource:0}: Error finding container 79fe7ed9a238c62401b1f71acf077bdcd644124e17266bf1b80c3a49debd89e0: Status 404 returned error can't find the container with id 79fe7ed9a238c62401b1f71acf077bdcd644124e17266bf1b80c3a49debd89e0 Jul 29 12:26:35.417643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.417616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lpk5s"] Jul 29 12:26:35.420351 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:35.420321 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1972259_e332_4fbb_b3db_247f6bdb3f95.slice/crio-4164bc26cae3572adb4daf435ea62accd17740449d15de4fd347b688002f58b9 WatchSource:0}: Error finding container 4164bc26cae3572adb4daf435ea62accd17740449d15de4fd347b688002f58b9: Status 404 returned error can't find the container with id 4164bc26cae3572adb4daf435ea62accd17740449d15de4fd347b688002f58b9 Jul 29 12:26:35.940990 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.940629 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmgdg" event={"ID":"7b259a3c-d681-4dab-a491-38acf064c970","Type":"ContainerStarted","Data":"fb59a797904b466aa6605ddcafae80b7cf18ee61a8f31fd073cbf4daa3d211e0"} Jul 29 12:26:35.942935 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.942723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" event={"ID":"58aa9529-5196-4b4c-98de-3b7469d0a375","Type":"ContainerStarted","Data":"5bbba9ab74d5cc1a513b0845ed595c6876ea19b5809f2c233a90f427f9f39747"} Jul 29 12:26:35.944332 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.944307 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lpk5s" event={"ID":"a1972259-e332-4fbb-b3db-247f6bdb3f95","Type":"ContainerStarted","Data":"4164bc26cae3572adb4daf435ea62accd17740449d15de4fd347b688002f58b9"} Jul 29 12:26:35.946865 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.946835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" event={"ID":"09aa5fa8-e30b-432d-b07c-5c71d21f241b","Type":"ContainerStarted","Data":"26c6f604ab31283fe835b32bcc086c40ce721cf3cd38cb16673427c051e4be62"} Jul 29 12:26:35.947253 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.946869 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" event={"ID":"09aa5fa8-e30b-432d-b07c-5c71d21f241b","Type":"ContainerStarted","Data":"c6c5823edeb42873ba08cce9ea0627f332d7cfda24f6dc851b80eaf68efa98a4"} Jul 29 12:26:35.947253 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.946886 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" event={"ID":"09aa5fa8-e30b-432d-b07c-5c71d21f241b","Type":"ContainerStarted","Data":"0434570ffbc34dfbebfc97780a1c5d8ae65d46b981c056797f27f2ab13902298"} Jul 29 12:26:35.948623 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.948590 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zcf8c" event={"ID":"7d7e341b-4606-4c63-a278-9f9d71a68296","Type":"ContainerStarted","Data":"733a3e55bc5cddaabd113e40dcda2fde5454f3913fc067c71573e566d50a37ab"} Jul 29 12:26:35.949899 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.949873 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7m5ln" event={"ID":"9562d83d-4984-4e18-be4c-c37408d9169c","Type":"ContainerStarted","Data":"79fe7ed9a238c62401b1f71acf077bdcd644124e17266bf1b80c3a49debd89e0"} Jul 29 12:26:35.963024 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.962978 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cmgdg" podStartSLOduration=17.026484604 podStartE2EDuration="31.962960649s" totalImagesPullingTime="14.936476045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:26:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:35.961526862 +0000 UTC m=+68.864723536" watchObservedRunningTime="2026-07-29 12:26:35.962960649 +0000 UTC m=+68.866157327" Jul 29 12:26:35.981699 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.981645 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6889447bf9-fxthh" podStartSLOduration=2.235203587 podStartE2EDuration="7.981628092s" totalImagesPullingTime="5.746424505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:35.979968932 +0000 UTC m=+68.883165609" watchObservedRunningTime="2026-07-29 12:26:35.981628092 +0000 UTC m=+68.884824768" Jul 29 12:26:36.001446 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:35.996980 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-659786d874-vbfj6" podStartSLOduration=2.22000357 podStartE2EDuration="7.996963578s" totalImagesPullingTime="5.776960008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:35.996043121 +0000 UTC m=+68.899239797" watchObservedRunningTime="2026-07-29 12:26:35.996963578 +0000 UTC m=+68.900160256" Jul 29 12:26:36.956269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:36.956185 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zcf8c" event={"ID":"7d7e341b-4606-4c63-a278-9f9d71a68296","Type":"ContainerStarted","Data":"5ea503b45a18cc8daae6189df6894e3f1a64e4408a1604ca9a1b756125dc3120"} Jul 29 12:26:36.956715 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:36.956285 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zcf8c" event={"ID":"7d7e341b-4606-4c63-a278-9f9d71a68296","Type":"ContainerStarted","Data":"b41203abc5141f05b4ba4627e3f010b4ed3f97ee33c8e8cb28cb267c8451a829"} Jul 29 12:26:36.976294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:36.976244 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zcf8c" podStartSLOduration=68.905787087 podStartE2EDuration="1m9.976226151s" totalImagesPullingTime="1.070439064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:36.974810928 +0000 UTC m=+69.878007604" watchObservedRunningTime="2026-07-29 12:26:36.976226151 +0000 UTC m=+69.879422830" Jul 29 12:26:37.099002 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:37.098941 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:37.176822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:37.176789 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:38.039861 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:38.039813 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:38.138143 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:38.138109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:38.815984 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:38.815952 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:38.965308 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:38.965246 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7m5ln" event={"ID":"9562d83d-4984-4e18-be4c-c37408d9169c","Type":"ContainerStarted","Data":"c792c2f7db1cbc771d2f718a549e16a0ac07d087979bc06547a8ffde0440552d"} Jul 29 12:26:38.965489 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:38.965389 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:26:38.966017 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:38.965993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:39.084247 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:39.084174 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:39.462262 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:39.462202 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:39.559304 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:39.559271 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:40.104183 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.104146 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:40.205178 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.205108 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:40.420672 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.420638 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:40.490137 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.490109 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:40.978760 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.978730 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lpk5s" event={"ID":"a1972259-e332-4fbb-b3db-247f6bdb3f95","Type":"ContainerStarted","Data":"82b2774d02bbcb1acdd63aff12e082ff41b64095483f84be0d42a19d9a41d957"} Jul 29 12:26:40.995950 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.995898 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-lpk5s" podStartSLOduration=69.520170925 podStartE2EDuration="1m13.995882731s" totalImagesPullingTime="4.475711806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:40.995232584 +0000 UTC m=+73.898429263" watchObservedRunningTime="2026-07-29 12:26:40.995882731 +0000 UTC m=+73.899079406" Jul 29 12:26:40.996066 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:40.996016 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7m5ln" podStartSLOduration=71.072510568 podStartE2EDuration="1m13.996010931s" totalImagesPullingTime="2.923500363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:38.997114234 +0000 UTC m=+71.900310913" watchObservedRunningTime="2026-07-29 12:26:40.996010931 +0000 UTC m=+73.899207607" Jul 29 12:26:44.079991 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:44.079959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:44.218947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:44.218915 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:44.309018 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:44.308993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:44.405294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:44.405208 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:45.202143 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:45.202117 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:45.301876 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:45.301849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:46.298879 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:46.298849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:47.377605 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:47.377572 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:47.479384 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:47.479359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:48.755159 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:48.755130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:49.028720 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.028651 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:49.261494 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.261451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:49.565892 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.565860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:49.691784 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.691757 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8b4k9"] Jul 29 12:26:49.696273 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.696253 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.699285 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.699263 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 12:26:49.699535 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.699520 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 12:26:49.699786 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.699770 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 12:26:49.699867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.699844 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 12:26:49.700426 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.700406 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 12:26:49.700509 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.700413 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 12:26:49.700509 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.700441 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wzq9d\"" Jul 29 12:26:49.791274 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791251 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-wtmp\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791281 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-textfile\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791303 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-metrics-client-ca\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-root\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791443 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm5gh\" (UniqueName: \"kubernetes.io/projected/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-kube-api-access-nm5gh\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791481 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-sys\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.791622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.791517 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-tls\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.886890 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.886830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:49.892047 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892029 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-metrics-client-ca\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892100 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-root\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892119 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm5gh\" (UniqueName: \"kubernetes.io/projected/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-kube-api-access-nm5gh\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-sys\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892162 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-tls\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892179 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-wtmp\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-textfile\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:49.892283 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:26:49.892338 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-tls podName:7b0b1a82-9965-4818-9ea7-ceb58b58a0a6 nodeName:}" failed. No retries permitted until 2026-07-29 12:26:50.392321689 +0000 UTC m=+83.295518349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-tls") pod "node-exporter-8b4k9" (UID: "7b0b1a82-9965-4818-9ea7-ceb58b58a0a6") : secret "node-exporter-tls" not found Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892348 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-sys\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892399 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-root\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892761 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892422 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-wtmp\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892761 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892713 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-textfile\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892761 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-metrics-client-ca\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.892949 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.892928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.894389 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.894363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.901085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.901061 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm5gh\" (UniqueName: \"kubernetes.io/projected/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-kube-api-access-nm5gh\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:49.997389 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:49.997367 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:50.189848 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.189779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:50.297163 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.297137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:50.394985 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.394965 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-tls\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:50.397010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.396994 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b0b1a82-9965-4818-9ea7-ceb58b58a0a6-node-exporter-tls\") pod \"node-exporter-8b4k9\" (UID: \"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6\") " pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:50.456871 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.456819 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:50.605371 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.605345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8b4k9" Jul 29 12:26:50.615339 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:50.615307 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b0b1a82_9965_4818_9ea7_ceb58b58a0a6.slice/crio-02509be6b110427346fedcf6c9fa0ab3d77a231bf41bfca919943e4a1e2bdc25 WatchSource:0}: Error finding container 02509be6b110427346fedcf6c9fa0ab3d77a231bf41bfca919943e4a1e2bdc25: Status 404 returned error can't find the container with id 02509be6b110427346fedcf6c9fa0ab3d77a231bf41bfca919943e4a1e2bdc25 Jul 29 12:26:50.621891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.621851 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:50.773003 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.772985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:50.928210 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:50.928192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:51.014210 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:51.014177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8b4k9" event={"ID":"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6","Type":"ContainerStarted","Data":"02509be6b110427346fedcf6c9fa0ab3d77a231bf41bfca919943e4a1e2bdc25"} Jul 29 12:26:51.186947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:51.186891 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:51.287302 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:51.287281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:51.810358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:51.810334 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:52.019045 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.019014 2563 generic.go:356] "Generic (PLEG): container finished" podID="7b0b1a82-9965-4818-9ea7-ceb58b58a0a6" containerID="b6747940879e9644055d3dc062f1dd6918e6d872942e916d45c7a1e0b0917340" exitCode=0 Jul 29 12:26:52.019366 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.019061 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8b4k9" event={"ID":"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6","Type":"ContainerDied","Data":"b6747940879e9644055d3dc062f1dd6918e6d872942e916d45c7a1e0b0917340"} Jul 29 12:26:52.273271 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.273240 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:52.436661 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.436598 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:52.532959 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.532939 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:52.643105 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.643084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:52.961155 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:52.961127 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:53.024247 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:53.024202 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8b4k9" event={"ID":"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6","Type":"ContainerStarted","Data":"27360933abd1eb55e5ff27f02047f8efd1e273656f60ed9632738646b7dd524a"} Jul 29 12:26:53.024624 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:53.024264 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8b4k9" event={"ID":"7b0b1a82-9965-4818-9ea7-ceb58b58a0a6","Type":"ContainerStarted","Data":"cb54e5514ea66c93167c51a330e6d51c92bf1dbf3f3fa61ef6fe898f83bde38c"} Jul 29 12:26:53.044434 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:53.044391 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8b4k9" podStartSLOduration=2.82362264 podStartE2EDuration="4.044376607s" totalImagesPullingTime="1.160779843s" totalInitContainerRuntime="59.974124ms" isStatefulPod=true podCreationTimestamp="2026-07-29 12:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:26:53.042998888 +0000 UTC m=+85.946195563" watchObservedRunningTime="2026-07-29 12:26:53.044376607 +0000 UTC m=+85.947573282" Jul 29 12:26:53.076901 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:53.076874 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:54.282707 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:54.282663 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:54.405886 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:54.405853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:54.483372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:54.483349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:55.299558 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.299532 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:55.399350 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.399324 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:55.833479 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.833435 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:26:55.834148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.834126 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 12:26:55.848923 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.848905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.851919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.851900 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 29 12:26:55.852036 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.851901 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 29 12:26:55.852036 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.851903 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 29 12:26:55.852036 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.851958 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 29 12:26:55.852704 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.852686 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 29 12:26:55.852791 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.852709 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 29 12:26:55.852791 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.852784 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 29 12:26:55.852903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.852808 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fsp1v5r8l35rq\"" Jul 29 12:26:55.852988 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.852974 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 29 12:26:55.853160 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.853135 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 29 12:26:55.853246 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.853158 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 29 12:26:55.853297 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.853271 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ff8h5\"" Jul 29 12:26:55.853335 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.853293 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 29 12:26:55.853587 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.853571 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 29 12:26:55.853816 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.853802 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 29 12:26:55.856958 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.856939 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 29 12:26:55.858559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.858540 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 12:26:55.859369 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.859353 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 29 12:26:55.934670 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934780 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934780 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934780 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934752 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934809 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934827 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934866 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.934891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934884 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934914 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934957 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934974 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.934995 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935013 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935028 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2k2g\" (UniqueName: \"kubernetes.io/projected/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-kube-api-access-h2k2g\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935296 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935296 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935123 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935296 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935140 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935296 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935154 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-config\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:55.935296 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:55.935182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.035673 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035647 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.035783 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035675 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.035783 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035692 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.035783 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035708 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.035783 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035731 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2k2g\" (UniqueName: \"kubernetes.io/projected/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-kube-api-access-h2k2g\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.035783 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035791 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035812 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035868 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-config\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035954 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.035982 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036020 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036012 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036043 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036102 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036162 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036191 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036215 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036729 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036431 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036729 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036434 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.036987 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.036964 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.039204 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.039177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.039488 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.039410 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.039768 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.039746 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.039885 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.039841 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040054 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040355 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040492 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040472 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040564 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040564 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040550 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-config\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040678 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040604 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.040842 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.041010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.040987 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.041246 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.041223 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.041352 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.041336 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.042241 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.042219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.042558 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.042543 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.044578 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.044559 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2k2g\" (UniqueName: \"kubernetes.io/projected/3e4bfa46-e9df-4ff6-8e13-21f7854eaffe-kube-api-access-h2k2g\") pod \"prometheus-k8s-0\" (UID: \"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.159709 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.159661 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:26:56.291934 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:56.291910 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 29 12:26:56.294031 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:26:56.294008 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e4bfa46_e9df_4ff6_8e13_21f7854eaffe.slice/crio-10b363793a5678da39cce6447ce61100c55e93f423ca34d2027a782e5b5e5c2d WatchSource:0}: Error finding container 10b363793a5678da39cce6447ce61100c55e93f423ca34d2027a782e5b5e5c2d: Status 404 returned error can't find the container with id 10b363793a5678da39cce6447ce61100c55e93f423ca34d2027a782e5b5e5c2d Jul 29 12:26:57.039346 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:57.039303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"10b363793a5678da39cce6447ce61100c55e93f423ca34d2027a782e5b5e5c2d"} Jul 29 12:26:58.044114 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:58.044076 2563 generic.go:356] "Generic (PLEG): container finished" podID="3e4bfa46-e9df-4ff6-8e13-21f7854eaffe" containerID="a2bd1e8d30502f202c54c394736baf94fdbb61d8cf323444c026b9e9e28340bc" exitCode=0 Jul 29 12:26:58.044541 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:26:58.044152 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerDied","Data":"a2bd1e8d30502f202c54c394736baf94fdbb61d8cf323444c026b9e9e28340bc"} Jul 29 12:27:00.599120 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:00.599087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:00.692794 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:00.692761 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:01.135862 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:01.135830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:01.236264 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:01.236229 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:01.427777 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:01.427682 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:01.545926 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:01.545889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:02.060476 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:02.060436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"020138f2d44d484e1999e76dcc94ad03f86c894b6aa399f29b9f48b26c4a03be"} Jul 29 12:27:02.060799 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:02.060511 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"41124d850665e2bb7971ed50afc0a7e1d06e45cffc0ec132c9c35bc52a6ba641"} Jul 29 12:27:04.070635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:04.070578 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"a0205146dfc7af94ff6c5d21c3f05d94cc4927264c041c2758a2d0213899f35c"} Jul 29 12:27:04.070926 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:04.070655 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"c7a51a332d1268f3d84ee1de4ed0f2261784f7f1fd5c6b740b049a91d553860c"} Jul 29 12:27:05.078254 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:05.078224 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"c3ec6f1e20bec4a85e7fdae449062e24d75c1c66944e3d6063563bb7550e1700"} Jul 29 12:27:05.078644 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:05.078287 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e4bfa46-e9df-4ff6-8e13-21f7854eaffe","Type":"ContainerStarted","Data":"61912ad8c79ba21af8f58527bcdfe907714c6922e472b23484bbfc9075f787e5"} Jul 29 12:27:05.112591 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:05.112545 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.616462414 podStartE2EDuration="10.112525231s" totalImagesPullingTime="6.478591033s" totalInitContainerRuntime="17.471784ms" isStatefulPod=false podCreationTimestamp="2026-07-29 12:26:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:27:05.110876733 +0000 UTC m=+98.014073408" watchObservedRunningTime="2026-07-29 12:27:05.112525231 +0000 UTC m=+98.015721909" Jul 29 12:27:05.641129 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:05.641097 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:05.746266 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:05.746235 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:05.805234 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:05.805189 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:06.159976 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:06.159954 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:27:09.975975 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:09.975938 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7m5ln" Jul 29 12:27:15.150189 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:15.150156 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4kxgm_d69591be-c441-4afe-8c26-97b512243e07/serve-healthcheck-canary/0.log" Jul 29 12:27:15.226299 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:15.226269 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:15.316904 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:15.316871 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:24.025902 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:24.025860 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:33.342885 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:33.342849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:38.193122 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:38.193090 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:38.299797 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:38.299767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:38.372809 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:38.372782 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:27:56.160122 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:56.160085 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:27:56.179643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:56.179617 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:27:56.265774 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:27:56.265745 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 29 12:28:08.913687 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:08.913650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:09.744669 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:09.744634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:18.677671 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:18.677631 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:18.736842 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:18.736810 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:24.260981 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:24.260891 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:24.349405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:24.349374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:25.902193 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:25.902162 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:25.963780 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:25.963749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:26.770731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:26.770697 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:26.871225 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:26.871199 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:36.196057 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:36.196022 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:36.258472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:36.258433 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:36.735153 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:36.735127 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:36.794350 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:36.794321 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:36.857781 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:36.857756 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:39.300307 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:39.300270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:45.198548 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:45.198518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:28:45.299589 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:28:45.299556 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:29:44.677349 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:29:44.677313 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:29:44.779494 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:29:44.779447 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:11.997872 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:11.997840 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:13.444113 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:13.444075 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:15.833381 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:15.833341 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:15.935158 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:15.935130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:16.034335 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:16.034307 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:16.791977 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:16.791943 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:18.077205 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:18.077172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:18.182171 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:18.182140 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:19.916774 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:19.916733 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:21.324922 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:21.324894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:22.597900 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:22.597869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:22.670571 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:22.670544 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:27.475328 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:27.475303 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 12:30:28.517444 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:28.517409 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:28.647652 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:28.647619 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:28.751814 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:28.751792 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:30.953058 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:30.953001 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:31.059649 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:31.059614 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:34.239398 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:34.239366 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:34.337217 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:34.337186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:36.454651 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:36.454606 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:46.681476 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.681414 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:46.735633 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.735605 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv"] Jul 29 12:30:46.738539 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.738525 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.745436 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.745355 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 29 12:30:46.745436 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.745369 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 29 12:30:46.745436 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.745396 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 29 12:30:46.745798 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.745431 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 29 12:30:46.745798 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.745376 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 29 12:30:46.745798 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.745487 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hw5pt\"" Jul 29 12:30:46.758441 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.758421 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv"] Jul 29 12:30:46.785943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.785920 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:46.878908 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.878886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58659516-7820-4a11-a02e-47e47a1e1eff-cert\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.879011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.878926 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx92n\" (UniqueName: \"kubernetes.io/projected/58659516-7820-4a11-a02e-47e47a1e1eff-kube-api-access-hx92n\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.879011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.878968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/58659516-7820-4a11-a02e-47e47a1e1eff-manager-config\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.879011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.878994 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/58659516-7820-4a11-a02e-47e47a1e1eff-metrics-cert\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.979511 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.979488 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx92n\" (UniqueName: \"kubernetes.io/projected/58659516-7820-4a11-a02e-47e47a1e1eff-kube-api-access-hx92n\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.979674 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.979535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/58659516-7820-4a11-a02e-47e47a1e1eff-manager-config\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.979674 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.979556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/58659516-7820-4a11-a02e-47e47a1e1eff-metrics-cert\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.979674 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.979579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58659516-7820-4a11-a02e-47e47a1e1eff-cert\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.980167 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.980148 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/58659516-7820-4a11-a02e-47e47a1e1eff-manager-config\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.981924 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.981904 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58659516-7820-4a11-a02e-47e47a1e1eff-cert\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.981993 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.981965 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/58659516-7820-4a11-a02e-47e47a1e1eff-metrics-cert\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:46.988288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:46.988268 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx92n\" (UniqueName: \"kubernetes.io/projected/58659516-7820-4a11-a02e-47e47a1e1eff-kube-api-access-hx92n\") pod \"lws-controller-manager-69d46b4769-lbwdv\" (UID: \"58659516-7820-4a11-a02e-47e47a1e1eff\") " pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:47.047068 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:47.047048 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:47.169368 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:47.169342 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv"] Jul 29 12:30:47.172145 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:30:47.172121 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58659516_7820_4a11_a02e_47e47a1e1eff.slice/crio-cc4205ecb5640e0680fb06ba6604073985460bc1fb36d69ea24eba2c41cfa20b WatchSource:0}: Error finding container cc4205ecb5640e0680fb06ba6604073985460bc1fb36d69ea24eba2c41cfa20b: Status 404 returned error can't find the container with id cc4205ecb5640e0680fb06ba6604073985460bc1fb36d69ea24eba2c41cfa20b Jul 29 12:30:47.174035 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:47.174019 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 12:30:47.834529 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:47.834498 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" event={"ID":"58659516-7820-4a11-a02e-47e47a1e1eff","Type":"ContainerStarted","Data":"cc4205ecb5640e0680fb06ba6604073985460bc1fb36d69ea24eba2c41cfa20b"} Jul 29 12:30:49.843662 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:49.843625 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" event={"ID":"58659516-7820-4a11-a02e-47e47a1e1eff","Type":"ContainerStarted","Data":"b01fe0a836438257b9e1d6191f22dbf671b0e05691ccbeda49e4ec469af50d5e"} Jul 29 12:30:49.844051 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:49.843764 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:30:51.059843 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:51.059809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:51.174301 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:51.174272 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:51.462043 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:51.462011 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.016345 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.016311 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.169023 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.168994 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.239294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.239252 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" podStartSLOduration=3.796368071 podStartE2EDuration="6.239237578s" totalImagesPullingTime="2.442869507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:49.895045146 +0000 UTC m=+322.798241822" watchObservedRunningTime="2026-07-29 12:30:52.239237578 +0000 UTC m=+325.142434253" Jul 29 12:30:52.239544 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.239529 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42"] Jul 29 12:30:52.242611 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.242597 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.254597 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.254579 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 29 12:30:52.254711 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.254603 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 29 12:30:52.254711 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.254672 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 29 12:30:52.254885 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.254871 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kbp6j\"" Jul 29 12:30:52.254923 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.254898 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 29 12:30:52.271176 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.271112 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42"] Jul 29 12:30:52.320618 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.320594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr8r9\" (UniqueName: \"kubernetes.io/projected/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-kube-api-access-nr8r9\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.320713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.320624 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.320713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.320643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-tmp\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.320713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.320660 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-manifests\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.320809 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.320755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-webhook-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.396613 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.396589 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.421768 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.421744 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr8r9\" (UniqueName: \"kubernetes.io/projected/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-kube-api-access-nr8r9\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.421845 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.421774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.421845 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.421795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-tmp\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.421845 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.421811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-manifests\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.421845 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.421839 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-webhook-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.422218 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.422199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-manifests\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.424155 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.424133 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-tmp\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.424258 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.424188 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.424312 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.424259 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-webhook-cert\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.431648 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.431628 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr8r9\" (UniqueName: \"kubernetes.io/projected/b4fa787f-e137-47ab-aac6-5236dc7e9bf7-kube-api-access-nr8r9\") pod \"opendatahub-operator-controller-manager-556f8bf875-j4c42\" (UID: \"b4fa787f-e137-47ab-aac6-5236dc7e9bf7\") " pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.495502 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.495479 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.551699 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.551647 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:52.597358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.597332 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.689250 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.689226 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42"] Jul 29 12:30:52.691280 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:30:52.691246 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fa787f_e137_47ab_aac6_5236dc7e9bf7.slice/crio-f87d3d3a54076203b91546559905273baadbbcb7322ebff815be7002e6cefbf7 WatchSource:0}: Error finding container f87d3d3a54076203b91546559905273baadbbcb7322ebff815be7002e6cefbf7: Status 404 returned error can't find the container with id f87d3d3a54076203b91546559905273baadbbcb7322ebff815be7002e6cefbf7 Jul 29 12:30:52.755500 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.755450 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.841276 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.841211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:30:52.854857 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:52.854835 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" event={"ID":"b4fa787f-e137-47ab-aac6-5236dc7e9bf7","Type":"ContainerStarted","Data":"f87d3d3a54076203b91546559905273baadbbcb7322ebff815be7002e6cefbf7"} Jul 29 12:30:55.869089 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:55.869046 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" event={"ID":"b4fa787f-e137-47ab-aac6-5236dc7e9bf7","Type":"ContainerStarted","Data":"38ef5dcc5c65d90066e18ea69469cb627fa64e5bcbe55cc896d69875b9edbcde"} Jul 29 12:30:56.874369 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:56.874333 2563 generic.go:356] "Generic (PLEG): container finished" podID="b4fa787f-e137-47ab-aac6-5236dc7e9bf7" containerID="38ef5dcc5c65d90066e18ea69469cb627fa64e5bcbe55cc896d69875b9edbcde" exitCode=0 Jul 29 12:30:56.874688 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:56.874401 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" event={"ID":"b4fa787f-e137-47ab-aac6-5236dc7e9bf7","Type":"ContainerDied","Data":"38ef5dcc5c65d90066e18ea69469cb627fa64e5bcbe55cc896d69875b9edbcde"} Jul 29 12:30:57.881505 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:57.881447 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" event={"ID":"b4fa787f-e137-47ab-aac6-5236dc7e9bf7","Type":"ContainerStarted","Data":"844979fb350ac4d6a90f3eed1d35bf5296d300965bfcaf96bba5a78884929199"} Jul 29 12:30:57.881947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:57.881907 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:30:57.907903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:30:57.907856 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" podStartSLOduration=1.192372899 podStartE2EDuration="5.907840656s" totalImagesPullingTime="3.763086143s" totalInitContainerRuntime="952.381614ms" isStatefulPod=false podCreationTimestamp="2026-07-29 12:30:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:30:57.907001157 +0000 UTC m=+330.810197833" watchObservedRunningTime="2026-07-29 12:30:57.907840656 +0000 UTC m=+330.811037331" Jul 29 12:31:00.850261 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:00.850221 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69d46b4769-lbwdv" Jul 29 12:31:08.886449 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:08.886413 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-556f8bf875-j4c42" Jul 29 12:31:11.287504 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:11.287475 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:11.464857 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:11.464832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:12.308260 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:12.308227 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:12.548500 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:12.548469 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:12.737185 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:12.737154 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:13.417212 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:13.417152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:13.521226 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:13.521191 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:22.413673 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:22.413598 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:22.740706 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:22.740671 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:23.108192 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:23.108101 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:23.243940 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:23.243902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:23.402919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:23.402850 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:23.614077 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:23.614050 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:24.476343 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:24.476300 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:24.670648 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:24.670624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:26.130143 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:26.130110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:26.249269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:26.249241 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:27.998139 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:27.998106 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:31.593300 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.593262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:31.692280 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.692247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:31.809228 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.809199 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq"] Jul 29 12:31:31.813811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.813790 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.818417 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.818378 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 29 12:31:31.818417 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.818399 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 29 12:31:31.818417 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.818406 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-72wdw\"" Jul 29 12:31:31.818417 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.818378 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 29 12:31:31.827754 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.827733 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq"] Jul 29 12:31:31.857302 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.857243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:31.907642 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907762 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907646 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907762 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvfsm\" (UniqueName: \"kubernetes.io/projected/90d1cb53-be29-43d4-b209-15b60beaef6a-kube-api-access-kvfsm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907762 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907727 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907876 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907876 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907814 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907876 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907875 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:31.907963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:31.907919 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d1cb53-be29-43d4-b209-15b60beaef6a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.008985 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009123 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009123 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009035 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009123 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d1cb53-be29-43d4-b209-15b60beaef6a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009130 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009200 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvfsm\" (UniqueName: \"kubernetes.io/projected/90d1cb53-be29-43d4-b209-15b60beaef6a-kube-api-access-kvfsm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009231 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009259 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009524 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009388 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009524 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009428 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009624 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009679 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.009760 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.009742 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d1cb53-be29-43d4-b209-15b60beaef6a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.011486 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.011452 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.011601 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.011585 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.020023 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.020001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d1cb53-be29-43d4-b209-15b60beaef6a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.020214 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.020196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvfsm\" (UniqueName: \"kubernetes.io/projected/90d1cb53-be29-43d4-b209-15b60beaef6a-kube-api-access-kvfsm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq\" (UID: \"90d1cb53-be29-43d4-b209-15b60beaef6a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.061452 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.061427 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:32.124289 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.124245 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:32.201115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.201059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:31:32.274187 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:32.274140 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq"] Jul 29 12:31:32.276535 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:31:32.276506 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d1cb53_be29_43d4_b209_15b60beaef6a.slice/crio-f14b144360e3bf69018adb4f5b8f05597cf88a79ff825b75e1005d52a8a65e3f WatchSource:0}: Error finding container f14b144360e3bf69018adb4f5b8f05597cf88a79ff825b75e1005d52a8a65e3f: Status 404 returned error can't find the container with id f14b144360e3bf69018adb4f5b8f05597cf88a79ff825b75e1005d52a8a65e3f Jul 29 12:31:33.005555 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:33.005514 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" event={"ID":"90d1cb53-be29-43d4-b209-15b60beaef6a","Type":"ContainerStarted","Data":"f14b144360e3bf69018adb4f5b8f05597cf88a79ff825b75e1005d52a8a65e3f"} Jul 29 12:31:34.822200 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:34.822162 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:31:34.822443 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:34.822237 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:31:34.822443 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:34.822264 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:31:35.015047 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:35.015015 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" event={"ID":"90d1cb53-be29-43d4-b209-15b60beaef6a","Type":"ContainerStarted","Data":"99a930306aa1089108510600c254e7ee785bc8896ec3a96d9891260309370c5a"} Jul 29 12:31:35.040214 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:35.040169 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" podStartSLOduration=1.49672334 podStartE2EDuration="4.040156055s" totalImagesPullingTime="2.543432715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:31:35.038806164 +0000 UTC m=+367.942003033" watchObservedRunningTime="2026-07-29 12:31:35.040156055 +0000 UTC m=+367.943352730" Jul 29 12:31:35.125243 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:35.125181 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:35.129364 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:35.129340 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:36.018484 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:36.018434 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:31:36.019324 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:31:36.019307 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq" Jul 29 12:32:04.584129 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:04.584089 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:09.307731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:09.307675 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:09.399616 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:09.399579 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:12.993784 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:12.993752 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:13.099011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.098981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:13.142731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.142698 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j7rt8"] Jul 29 12:32:13.149658 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.149633 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:13.153344 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.153317 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zngpf\"" Jul 29 12:32:13.153664 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.153646 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 12:32:13.156637 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.156295 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 12:32:13.160009 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.159985 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j7rt8"] Jul 29 12:32:13.220856 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.220820 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:13.321872 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.321778 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-266hh\" (UniqueName: \"kubernetes.io/projected/3a0f544d-ab0e-4986-8f72-80197e1e9717-kube-api-access-266hh\") pod \"kuadrant-operator-catalog-j7rt8\" (UID: \"3a0f544d-ab0e-4986-8f72-80197e1e9717\") " pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:13.422977 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.422943 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-266hh\" (UniqueName: \"kubernetes.io/projected/3a0f544d-ab0e-4986-8f72-80197e1e9717-kube-api-access-266hh\") pod \"kuadrant-operator-catalog-j7rt8\" (UID: \"3a0f544d-ab0e-4986-8f72-80197e1e9717\") " pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:13.432453 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.432424 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-266hh\" (UniqueName: \"kubernetes.io/projected/3a0f544d-ab0e-4986-8f72-80197e1e9717-kube-api-access-266hh\") pod \"kuadrant-operator-catalog-j7rt8\" (UID: \"3a0f544d-ab0e-4986-8f72-80197e1e9717\") " pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:13.460294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.460261 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:13.467848 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.467818 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j7rt8"] Jul 29 12:32:13.590850 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.590817 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j7rt8"] Jul 29 12:32:13.593131 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:32:13.593104 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a0f544d_ab0e_4986_8f72_80197e1e9717.slice/crio-6c5f05be14dc3f2ea76954898d7ef323563f17038aa7a831d5f8956bc5524426 WatchSource:0}: Error finding container 6c5f05be14dc3f2ea76954898d7ef323563f17038aa7a831d5f8956bc5524426: Status 404 returned error can't find the container with id 6c5f05be14dc3f2ea76954898d7ef323563f17038aa7a831d5f8956bc5524426 Jul 29 12:32:13.698812 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.698775 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6ggkz"] Jul 29 12:32:13.703054 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.703029 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:13.719701 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.719671 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6ggkz"] Jul 29 12:32:13.826312 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.826273 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw25p\" (UniqueName: \"kubernetes.io/projected/2ffe3780-dcad-4f42-8c7e-f102785b5f15-kube-api-access-kw25p\") pod \"kuadrant-operator-catalog-6ggkz\" (UID: \"2ffe3780-dcad-4f42-8c7e-f102785b5f15\") " pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:13.927017 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.926929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw25p\" (UniqueName: \"kubernetes.io/projected/2ffe3780-dcad-4f42-8c7e-f102785b5f15-kube-api-access-kw25p\") pod \"kuadrant-operator-catalog-6ggkz\" (UID: \"2ffe3780-dcad-4f42-8c7e-f102785b5f15\") " pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:13.939753 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:13.939727 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw25p\" (UniqueName: \"kubernetes.io/projected/2ffe3780-dcad-4f42-8c7e-f102785b5f15-kube-api-access-kw25p\") pod \"kuadrant-operator-catalog-6ggkz\" (UID: \"2ffe3780-dcad-4f42-8c7e-f102785b5f15\") " pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:14.012677 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:14.012638 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:14.143665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:14.143636 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-6ggkz"] Jul 29 12:32:14.146095 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:32:14.146068 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ffe3780_dcad_4f42_8c7e_f102785b5f15.slice/crio-5e201620dce855a7d633f79cebef45b3646da8e57c0cb590822b7e2791461780 WatchSource:0}: Error finding container 5e201620dce855a7d633f79cebef45b3646da8e57c0cb590822b7e2791461780: Status 404 returned error can't find the container with id 5e201620dce855a7d633f79cebef45b3646da8e57c0cb590822b7e2791461780 Jul 29 12:32:14.152143 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:14.152113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" event={"ID":"3a0f544d-ab0e-4986-8f72-80197e1e9717","Type":"ContainerStarted","Data":"6c5f05be14dc3f2ea76954898d7ef323563f17038aa7a831d5f8956bc5524426"} Jul 29 12:32:14.153166 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:14.153144 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" event={"ID":"2ffe3780-dcad-4f42-8c7e-f102785b5f15","Type":"ContainerStarted","Data":"5e201620dce855a7d633f79cebef45b3646da8e57c0cb590822b7e2791461780"} Jul 29 12:32:16.163331 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.163229 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" event={"ID":"3a0f544d-ab0e-4986-8f72-80197e1e9717","Type":"ContainerStarted","Data":"0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce"} Jul 29 12:32:16.163331 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.163248 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" podUID="3a0f544d-ab0e-4986-8f72-80197e1e9717" containerName="registry-server" containerID="cri-o://0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce" gracePeriod=2 Jul 29 12:32:16.164823 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.164788 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" event={"ID":"2ffe3780-dcad-4f42-8c7e-f102785b5f15","Type":"ContainerStarted","Data":"cc0643e2d7be40adb809cf72d6c7b8843cae1a945d86a3e0936479e8e75f1034"} Jul 29 12:32:16.183646 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.183598 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" podStartSLOduration=0.865788074 podStartE2EDuration="3.183580507s" totalImagesPullingTime="2.317792433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:32:16.182682169 +0000 UTC m=+409.085878847" watchObservedRunningTime="2026-07-29 12:32:16.183580507 +0000 UTC m=+409.086777183" Jul 29 12:32:16.202728 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.202673 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" podStartSLOduration=1.441299928 podStartE2EDuration="3.202653866s" totalImagesPullingTime="1.761353938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:32:16.202297512 +0000 UTC m=+409.105494212" watchObservedRunningTime="2026-07-29 12:32:16.202653866 +0000 UTC m=+409.105850542" Jul 29 12:32:16.403692 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.403666 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:16.548427 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.548394 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-266hh\" (UniqueName: \"kubernetes.io/projected/3a0f544d-ab0e-4986-8f72-80197e1e9717-kube-api-access-266hh\") pod \"3a0f544d-ab0e-4986-8f72-80197e1e9717\" (UID: \"3a0f544d-ab0e-4986-8f72-80197e1e9717\") " Jul 29 12:32:16.550580 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.550546 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a0f544d-ab0e-4986-8f72-80197e1e9717-kube-api-access-266hh" (OuterVolumeSpecName: "kube-api-access-266hh") pod "3a0f544d-ab0e-4986-8f72-80197e1e9717" (UID: "3a0f544d-ab0e-4986-8f72-80197e1e9717"). InnerVolumeSpecName "kube-api-access-266hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:32:16.649170 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:16.649134 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-266hh\" (UniqueName: \"kubernetes.io/projected/3a0f544d-ab0e-4986-8f72-80197e1e9717-kube-api-access-266hh\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:32:17.169288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.169253 2563 generic.go:356] "Generic (PLEG): container finished" podID="3a0f544d-ab0e-4986-8f72-80197e1e9717" containerID="0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce" exitCode=0 Jul 29 12:32:17.169720 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.169300 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" Jul 29 12:32:17.169720 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.169343 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" event={"ID":"3a0f544d-ab0e-4986-8f72-80197e1e9717","Type":"ContainerDied","Data":"0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce"} Jul 29 12:32:17.169720 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.169394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j7rt8" event={"ID":"3a0f544d-ab0e-4986-8f72-80197e1e9717","Type":"ContainerDied","Data":"6c5f05be14dc3f2ea76954898d7ef323563f17038aa7a831d5f8956bc5524426"} Jul 29 12:32:17.169720 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.169419 2563 scope.go:117] "RemoveContainer" containerID="0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce" Jul 29 12:32:17.178992 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.178965 2563 scope.go:117] "RemoveContainer" containerID="0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce" Jul 29 12:32:17.179243 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:32:17.179225 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce\": container with ID starting with 0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce not found: ID does not exist" containerID="0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce" Jul 29 12:32:17.179298 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.179253 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce"} err="failed to get container status \"0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce\": rpc error: code = NotFound desc = could not find container \"0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce\": container with ID starting with 0e5156837fa21034d6c0d4605cb0d553c413120809584f6ed617bf39f5339bce not found: ID does not exist" Jul 29 12:32:17.192524 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.192500 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j7rt8"] Jul 29 12:32:17.202567 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.202544 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j7rt8"] Jul 29 12:32:17.656447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:17.656412 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a0f544d-ab0e-4986-8f72-80197e1e9717" path="/var/lib/kubelet/pods/3a0f544d-ab0e-4986-8f72-80197e1e9717/volumes" Jul 29 12:32:24.013703 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:24.013668 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:24.014116 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:24.013724 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:24.034763 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:24.034735 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:24.217251 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:24.217223 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-6ggkz" Jul 29 12:32:39.081995 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:39.081957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:42.658409 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:42.658376 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:45.317957 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.317920 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9"] Jul 29 12:32:45.318325 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.318196 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a0f544d-ab0e-4986-8f72-80197e1e9717" containerName="registry-server" Jul 29 12:32:45.318325 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.318206 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a0f544d-ab0e-4986-8f72-80197e1e9717" containerName="registry-server" Jul 29 12:32:45.318325 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.318258 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a0f544d-ab0e-4986-8f72-80197e1e9717" containerName="registry-server" Jul 29 12:32:45.322521 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.322502 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:32:45.329712 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.329689 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qb8s6\"" Jul 29 12:32:45.329811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.329695 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 29 12:32:45.338079 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.338054 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9"] Jul 29 12:32:45.375703 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.375665 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p47qg\" (UniqueName: \"kubernetes.io/projected/5164145e-115e-401f-b187-a31e3a6830a8-kube-api-access-p47qg\") pod \"dns-operator-controller-manager-864dd78c4c-c8ss9\" (UID: \"5164145e-115e-401f-b187-a31e3a6830a8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:32:45.476801 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.476761 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p47qg\" (UniqueName: \"kubernetes.io/projected/5164145e-115e-401f-b187-a31e3a6830a8-kube-api-access-p47qg\") pod \"dns-operator-controller-manager-864dd78c4c-c8ss9\" (UID: \"5164145e-115e-401f-b187-a31e3a6830a8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:32:45.506427 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.506391 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p47qg\" (UniqueName: \"kubernetes.io/projected/5164145e-115e-401f-b187-a31e3a6830a8-kube-api-access-p47qg\") pod \"dns-operator-controller-manager-864dd78c4c-c8ss9\" (UID: \"5164145e-115e-401f-b187-a31e3a6830a8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:32:45.632317 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.632222 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:32:45.769480 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:45.769426 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9"] Jul 29 12:32:45.772579 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:32:45.772556 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5164145e_115e_401f_b187_a31e3a6830a8.slice/crio-baa2922620db0a393e724ec7b981ee92919c6d7cfa951feedb1a95d491dcac3c WatchSource:0}: Error finding container baa2922620db0a393e724ec7b981ee92919c6d7cfa951feedb1a95d491dcac3c: Status 404 returned error can't find the container with id baa2922620db0a393e724ec7b981ee92919c6d7cfa951feedb1a95d491dcac3c Jul 29 12:32:46.100811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:46.100781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:46.276847 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:46.276814 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" event={"ID":"5164145e-115e-401f-b187-a31e3a6830a8","Type":"ContainerStarted","Data":"baa2922620db0a393e724ec7b981ee92919c6d7cfa951feedb1a95d491dcac3c"} Jul 29 12:32:46.921882 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:46.921848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:49.297252 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.297211 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" event={"ID":"5164145e-115e-401f-b187-a31e3a6830a8","Type":"ContainerStarted","Data":"6c0504f89743d69d47270328979edaedf59431eb5c5805a764655109c8df2c56"} Jul 29 12:32:49.297639 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.297344 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:32:49.328691 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.328638 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" podStartSLOduration=1.7301466460000001 podStartE2EDuration="4.328621438s" totalImagesPullingTime="2.598474792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:32:49.328325322 +0000 UTC m=+442.231521999" watchObservedRunningTime="2026-07-29 12:32:49.328621438 +0000 UTC m=+442.231818115" Jul 29 12:32:49.586631 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.586519 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-c6srg"] Jul 29 12:32:49.593959 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.593938 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:32:49.598134 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.598110 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-9jznh\"" Jul 29 12:32:49.610978 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.610946 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-c6srg"] Jul 29 12:32:49.713936 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.713888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7w75\" (UniqueName: \"kubernetes.io/projected/df8bb46a-1aeb-493d-bee8-06f57bebcb0e-kube-api-access-r7w75\") pod \"authorino-operator-75cc8d5cb7-c6srg\" (UID: \"df8bb46a-1aeb-493d-bee8-06f57bebcb0e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:32:49.815363 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.815319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7w75\" (UniqueName: \"kubernetes.io/projected/df8bb46a-1aeb-493d-bee8-06f57bebcb0e-kube-api-access-r7w75\") pod \"authorino-operator-75cc8d5cb7-c6srg\" (UID: \"df8bb46a-1aeb-493d-bee8-06f57bebcb0e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:32:49.828060 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.828027 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7w75\" (UniqueName: \"kubernetes.io/projected/df8bb46a-1aeb-493d-bee8-06f57bebcb0e-kube-api-access-r7w75\") pod \"authorino-operator-75cc8d5cb7-c6srg\" (UID: \"df8bb46a-1aeb-493d-bee8-06f57bebcb0e\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:32:49.903734 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:49.903641 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:32:50.047307 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:50.047139 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-c6srg"] Jul 29 12:32:50.050091 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:32:50.050061 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8bb46a_1aeb_493d_bee8_06f57bebcb0e.slice/crio-f8681ec70c540a8627c84b09552cbfd5b1773e46bb8c22e7d7041962ddcee355 WatchSource:0}: Error finding container f8681ec70c540a8627c84b09552cbfd5b1773e46bb8c22e7d7041962ddcee355: Status 404 returned error can't find the container with id f8681ec70c540a8627c84b09552cbfd5b1773e46bb8c22e7d7041962ddcee355 Jul 29 12:32:50.302422 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:50.302384 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" event={"ID":"df8bb46a-1aeb-493d-bee8-06f57bebcb0e","Type":"ContainerStarted","Data":"f8681ec70c540a8627c84b09552cbfd5b1773e46bb8c22e7d7041962ddcee355"} Jul 29 12:32:50.535000 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:50.534965 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:50.662694 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:50.662604 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:50.780639 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:50.780576 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:52.513065 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:52.513033 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:53.322035 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:53.321998 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" event={"ID":"df8bb46a-1aeb-493d-bee8-06f57bebcb0e","Type":"ContainerStarted","Data":"376725023c916de896af3962c248eaf9f65a3f524f89945a6fcf070712f611c0"} Jul 29 12:32:53.322202 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:53.322104 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:32:53.345947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:53.345900 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" podStartSLOduration=2.04864525 podStartE2EDuration="4.345886347s" totalImagesPullingTime="2.297241097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:32:53.34494653 +0000 UTC m=+446.248143207" watchObservedRunningTime="2026-07-29 12:32:53.345886347 +0000 UTC m=+446.249083024" Jul 29 12:32:53.496269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:53.496233 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:53.909087 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:53.909055 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:55.075841 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:55.075796 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:55.639091 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:55.639054 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:57.702374 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.702336 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8"] Jul 29 12:32:57.704664 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.704642 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:57.707829 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.707807 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9h9ms\"" Jul 29 12:32:57.731055 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.731029 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8"] Jul 29 12:32:57.786435 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.786395 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/463a6e96-8475-4ae0-9936-8bf634bf0915-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:57.786435 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.786441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vcl9\" (UniqueName: \"kubernetes.io/projected/463a6e96-8475-4ae0-9936-8bf634bf0915-kube-api-access-7vcl9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:57.887037 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.886998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/463a6e96-8475-4ae0-9936-8bf634bf0915-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:57.887037 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.887039 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vcl9\" (UniqueName: \"kubernetes.io/projected/463a6e96-8475-4ae0-9936-8bf634bf0915-kube-api-access-7vcl9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:57.887448 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.887429 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/463a6e96-8475-4ae0-9936-8bf634bf0915-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:57.901641 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:57.901611 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vcl9\" (UniqueName: \"kubernetes.io/projected/463a6e96-8475-4ae0-9936-8bf634bf0915-kube-api-access-7vcl9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:58.017609 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:58.017573 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:32:58.148262 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:58.148230 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8"] Jul 29 12:32:58.151143 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:32:58.151114 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod463a6e96_8475_4ae0_9936_8bf634bf0915.slice/crio-f26c90fdf4d2ce8f72582e028b6703fec7203bb6fee1b87f6b0109844a2eab03 WatchSource:0}: Error finding container f26c90fdf4d2ce8f72582e028b6703fec7203bb6fee1b87f6b0109844a2eab03: Status 404 returned error can't find the container with id f26c90fdf4d2ce8f72582e028b6703fec7203bb6fee1b87f6b0109844a2eab03 Jul 29 12:32:58.343952 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:58.343861 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" event={"ID":"463a6e96-8475-4ae0-9936-8bf634bf0915","Type":"ContainerStarted","Data":"f26c90fdf4d2ce8f72582e028b6703fec7203bb6fee1b87f6b0109844a2eab03"} Jul 29 12:32:58.673186 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:58.673068 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:32:58.789390 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:32:58.789350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:00.304913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:00.304887 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-c8ss9" Jul 29 12:33:03.366982 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:03.366943 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" event={"ID":"463a6e96-8475-4ae0-9936-8bf634bf0915","Type":"ContainerStarted","Data":"1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e"} Jul 29 12:33:03.367324 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:03.367069 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:03.391962 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:03.390789 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" podStartSLOduration=1.4071410850000001 podStartE2EDuration="6.390770487s" totalImagesPullingTime="4.983629402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:33:03.389200435 +0000 UTC m=+456.292397112" watchObservedRunningTime="2026-07-29 12:33:03.390770487 +0000 UTC m=+456.293967165" Jul 29 12:33:04.047742 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:04.047703 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:04.145493 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:04.145442 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:04.329068 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:04.328988 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-c6srg" Jul 29 12:33:04.658915 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:04.658841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:04.789420 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:04.789383 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:06.014402 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:06.014368 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:06.184781 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:06.184754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:08.284136 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:08.284096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:08.383880 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:08.383849 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:10.651623 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:10.651591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:10.746920 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:10.746885 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:14.374295 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:14.374261 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:15.854957 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:15.854918 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl"] Jul 29 12:33:15.858228 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:15.858211 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:15.874196 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:15.874170 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl"] Jul 29 12:33:15.932798 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:15.932760 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:15.932983 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:15.932850 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbhl8\" (UniqueName: \"kubernetes.io/projected/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-kube-api-access-kbhl8\") pod \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.034247 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.034207 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.034405 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.034258 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbhl8\" (UniqueName: \"kubernetes.io/projected/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-kube-api-access-kbhl8\") pod \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.034610 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.034591 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.053665 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.053636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbhl8\" (UniqueName: \"kubernetes.io/projected/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-kube-api-access-kbhl8\") pod \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.168127 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.168048 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.306107 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.306081 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl"] Jul 29 12:33:16.308860 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:33:16.308831 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ab01a6_31b2_47e0_a2fc_9777fd7466d3.slice/crio-f9b0d181bfc3c45903fde2f22385915da4c16d60bf482c33ff3c85956f223279 WatchSource:0}: Error finding container f9b0d181bfc3c45903fde2f22385915da4c16d60bf482c33ff3c85956f223279: Status 404 returned error can't find the container with id f9b0d181bfc3c45903fde2f22385915da4c16d60bf482c33ff3c85956f223279 Jul 29 12:33:16.420540 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.420441 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" event={"ID":"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3","Type":"ContainerStarted","Data":"0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4"} Jul 29 12:33:16.420540 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.420510 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" event={"ID":"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3","Type":"ContainerStarted","Data":"f9b0d181bfc3c45903fde2f22385915da4c16d60bf482c33ff3c85956f223279"} Jul 29 12:33:16.420710 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.420587 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:16.453210 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.453123 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl"] Jul 29 12:33:16.464779 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.464748 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl"] Jul 29 12:33:16.513069 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.513024 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8"] Jul 29 12:33:16.513276 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.513248 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" containerName="manager" containerID="cri-o://1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e" gracePeriod=2 Jul 29 12:33:16.530495 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.530451 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8"] Jul 29 12:33:16.561607 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561574 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d"] Jul 29 12:33:16.561868 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561857 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" containerName="manager" Jul 29 12:33:16.561912 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561871 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" containerName="manager" Jul 29 12:33:16.561912 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561888 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" containerName="manager" Jul 29 12:33:16.561912 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561893 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" containerName="manager" Jul 29 12:33:16.561997 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561938 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" containerName="manager" Jul 29 12:33:16.561997 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.561947 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" containerName="manager" Jul 29 12:33:16.564852 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.564831 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.599554 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.599525 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d"] Jul 29 12:33:16.643044 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:16.642996 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:16.739398 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.739372 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zkj4d\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.739587 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.739431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k62j9\" (UniqueName: \"kubernetes.io/projected/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-kube-api-access-k62j9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zkj4d\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.742423 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.742404 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:16.744594 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:16.744567 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:16.840284 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.840249 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vcl9\" (UniqueName: \"kubernetes.io/projected/463a6e96-8475-4ae0-9936-8bf634bf0915-kube-api-access-7vcl9\") pod \"463a6e96-8475-4ae0-9936-8bf634bf0915\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " Jul 29 12:33:16.840483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.840346 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/463a6e96-8475-4ae0-9936-8bf634bf0915-extensions-socket-volume\") pod \"463a6e96-8475-4ae0-9936-8bf634bf0915\" (UID: \"463a6e96-8475-4ae0-9936-8bf634bf0915\") " Jul 29 12:33:16.840483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.840413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zkj4d\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.840483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.840450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k62j9\" (UniqueName: \"kubernetes.io/projected/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-kube-api-access-k62j9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zkj4d\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.840796 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.840771 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zkj4d\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.840898 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.840817 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/463a6e96-8475-4ae0-9936-8bf634bf0915-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "463a6e96-8475-4ae0-9936-8bf634bf0915" (UID: "463a6e96-8475-4ae0-9936-8bf634bf0915"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:33:16.842379 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.842357 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/463a6e96-8475-4ae0-9936-8bf634bf0915-kube-api-access-7vcl9" (OuterVolumeSpecName: "kube-api-access-7vcl9") pod "463a6e96-8475-4ae0-9936-8bf634bf0915" (UID: "463a6e96-8475-4ae0-9936-8bf634bf0915"). InnerVolumeSpecName "kube-api-access-7vcl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:33:16.853823 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.853804 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k62j9\" (UniqueName: \"kubernetes.io/projected/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-kube-api-access-k62j9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zkj4d\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.897770 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.897737 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:16.941174 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.941098 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/463a6e96-8475-4ae0-9936-8bf634bf0915-extensions-socket-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:33:16.941174 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:16.941130 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vcl9\" (UniqueName: \"kubernetes.io/projected/463a6e96-8475-4ae0-9936-8bf634bf0915-kube-api-access-7vcl9\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:33:17.040924 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.040890 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d"] Jul 29 12:33:17.044085 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:33:17.044055 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc6a63b_6aa3_45b2_bfad_07d2c4273e12.slice/crio-3ae8fff43fbb9ba80e8329329edd6c91cca6df9874b1c1498525673201f5c9e0 WatchSource:0}: Error finding container 3ae8fff43fbb9ba80e8329329edd6c91cca6df9874b1c1498525673201f5c9e0: Status 404 returned error can't find the container with id 3ae8fff43fbb9ba80e8329329edd6c91cca6df9874b1c1498525673201f5c9e0 Jul 29 12:33:17.426638 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.426604 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" event={"ID":"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12","Type":"ContainerStarted","Data":"a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7"} Jul 29 12:33:17.426809 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.426667 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" event={"ID":"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12","Type":"ContainerStarted","Data":"3ae8fff43fbb9ba80e8329329edd6c91cca6df9874b1c1498525673201f5c9e0"} Jul 29 12:33:17.426809 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.426800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:17.427856 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.427833 2563 generic.go:356] "Generic (PLEG): container finished" podID="463a6e96-8475-4ae0-9936-8bf634bf0915" containerID="1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e" exitCode=0 Jul 29 12:33:17.427986 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.427871 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:17.427986 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.427913 2563 scope.go:117] "RemoveContainer" containerID="1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e" Jul 29 12:33:17.428236 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.428219 2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" container="manager" Jul 29 12:33:17.436640 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.436625 2563 scope.go:117] "RemoveContainer" containerID="1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e" Jul 29 12:33:17.436935 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.436910 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e\": container with ID starting with 1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e not found: ID does not exist" containerID="1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e" Jul 29 12:33:17.437023 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.436940 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e"} err="failed to get container status \"1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e\": rpc error: code = NotFound desc = could not find container \"1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e\": container with ID starting with 1ad477f6d0c2d3a53fe38d89bbafb67534f8a53602235746c899155bab3cb36e not found: ID does not exist" Jul 29 12:33:17.439733 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.439706 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:17.460220 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.459922 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" podStartSLOduration=1.45990445 podStartE2EDuration="1.45990445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:33:17.458122947 +0000 UTC m=+470.361319624" watchObservedRunningTime="2026-07-29 12:33:17.45990445 +0000 UTC m=+470.363101128" Jul 29 12:33:17.460347 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.460278 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:17.469796 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.469770 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:17.472608 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.472580 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:17.591658 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.591624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:17.657275 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.657239 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-zspq8\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zspq8" Jul 29 12:33:17.658100 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:17.658077 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="463a6e96-8475-4ae0-9936-8bf634bf0915" path="/var/lib/kubelet/pods/463a6e96-8475-4ae0-9936-8bf634bf0915/volumes" Jul 29 12:33:17.659425 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:17.659397 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:18.433558 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.433516 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" containerName="manager" containerID="cri-o://0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4" gracePeriod=2 Jul 29 12:33:18.679443 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.679419 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:18.858215 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.858180 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbhl8\" (UniqueName: \"kubernetes.io/projected/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-kube-api-access-kbhl8\") pod \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " Jul 29 12:33:18.858372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.858266 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-extensions-socket-volume\") pod \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\" (UID: \"d0ab01a6-31b2-47e0-a2fc-9777fd7466d3\") " Jul 29 12:33:18.858541 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.858516 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" (UID: "d0ab01a6-31b2-47e0-a2fc-9777fd7466d3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:33:18.860175 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.860149 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-kube-api-access-kbhl8" (OuterVolumeSpecName: "kube-api-access-kbhl8") pod "d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" (UID: "d0ab01a6-31b2-47e0-a2fc-9777fd7466d3"). InnerVolumeSpecName "kube-api-access-kbhl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:33:18.959164 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.959129 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbhl8\" (UniqueName: \"kubernetes.io/projected/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-kube-api-access-kbhl8\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:33:18.959164 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:18.959159 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3-extensions-socket-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:33:19.440732 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:19.440697 2563 generic.go:356] "Generic (PLEG): container finished" podID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" containerID="0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4" exitCode=2 Jul 29 12:33:19.441148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:19.440739 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:19.441148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:19.440793 2563 scope.go:117] "RemoveContainer" containerID="0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4" Jul 29 12:33:19.443814 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:19.443785 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:19.449497 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:19.449478 2563 scope.go:117] "RemoveContainer" containerID="0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4" Jul 29 12:33:19.449739 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:19.449722 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4\": container with ID starting with 0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4 not found: ID does not exist" containerID="0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4" Jul 29 12:33:19.449780 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:19.449748 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4"} err="failed to get container status \"0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4\": rpc error: code = NotFound desc = could not find container \"0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4\": container with ID starting with 0af2799ace2c0e39545ffbd372ff36ad2ffe17c36f7273cbe8aa535ee3b008e4 not found: ID does not exist" Jul 29 12:33:19.452346 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:19.452325 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-4pndl\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-4pndl" Jul 29 12:33:19.657425 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:19.657393 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ab01a6-31b2-47e0-a2fc-9777fd7466d3" path="/var/lib/kubelet/pods/d0ab01a6-31b2-47e0-a2fc-9777fd7466d3/volumes" Jul 29 12:33:22.512121 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:22.512086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:22.614921 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:22.614888 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:22.726097 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:22.726063 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:22.826662 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:22.826593 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:25.079075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:25.079038 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:25.186032 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:25.186005 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:25.705997 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:25.705964 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:25.850019 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:25.849986 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:26.621819 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:26.621778 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:26.902089 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:26.902016 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:28.436610 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:28.436580 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:28.695310 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:28.695238 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:28.829619 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:28.829585 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:37.629016 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:37.628979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:37.727076 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:37.727040 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:37.815732 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:37.815699 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:33:43.904059 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:43.904027 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d"] Jul 29 12:33:43.904501 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:43.904254 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" podUID="1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" containerName="manager" containerID="cri-o://a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7" gracePeriod=10 Jul 29 12:33:44.143270 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.143248 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:44.197773 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.197694 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql"] Jul 29 12:33:44.198184 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.198142 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" containerName="manager" Jul 29 12:33:44.198184 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.198165 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" containerName="manager" Jul 29 12:33:44.198359 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.198250 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" containerName="manager" Jul 29 12:33:44.200146 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.200128 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.227271 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.227246 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql"] Jul 29 12:33:44.256943 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.256921 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-extensions-socket-volume\") pod \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " Jul 29 12:33:44.257036 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.256991 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k62j9\" (UniqueName: \"kubernetes.io/projected/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-kube-api-access-k62j9\") pod \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\" (UID: \"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12\") " Jul 29 12:33:44.257167 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.257148 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw264\" (UniqueName: \"kubernetes.io/projected/ed28d431-e95b-4be8-8bfa-3770bc321292-kube-api-access-rw264\") pod \"kuadrant-operator-controller-manager-86878c769f-48bql\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.257211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.257189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed28d431-e95b-4be8-8bfa-3770bc321292-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-48bql\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.257312 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.257295 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" (UID: "1bc6a63b-6aa3-45b2-bfad-07d2c4273e12"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:33:44.258881 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.258848 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-kube-api-access-k62j9" (OuterVolumeSpecName: "kube-api-access-k62j9") pod "1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" (UID: "1bc6a63b-6aa3-45b2-bfad-07d2c4273e12"). InnerVolumeSpecName "kube-api-access-k62j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:33:44.357912 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.357889 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw264\" (UniqueName: \"kubernetes.io/projected/ed28d431-e95b-4be8-8bfa-3770bc321292-kube-api-access-rw264\") pod \"kuadrant-operator-controller-manager-86878c769f-48bql\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.358025 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.357920 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed28d431-e95b-4be8-8bfa-3770bc321292-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-48bql\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.358025 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.357962 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k62j9\" (UniqueName: \"kubernetes.io/projected/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-kube-api-access-k62j9\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:33:44.358025 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.357973 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12-extensions-socket-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:33:44.358229 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.358211 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed28d431-e95b-4be8-8bfa-3770bc321292-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-48bql\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.370069 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.370049 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw264\" (UniqueName: \"kubernetes.io/projected/ed28d431-e95b-4be8-8bfa-3770bc321292-kube-api-access-rw264\") pod \"kuadrant-operator-controller-manager-86878c769f-48bql\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.510800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.510773 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:44.550364 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.550323 2563 generic.go:356] "Generic (PLEG): container finished" podID="1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" containerID="a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7" exitCode=0 Jul 29 12:33:44.550525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.550368 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" Jul 29 12:33:44.550525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.550401 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" event={"ID":"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12","Type":"ContainerDied","Data":"a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7"} Jul 29 12:33:44.550525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.550444 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d" event={"ID":"1bc6a63b-6aa3-45b2-bfad-07d2c4273e12","Type":"ContainerDied","Data":"3ae8fff43fbb9ba80e8329329edd6c91cca6df9874b1c1498525673201f5c9e0"} Jul 29 12:33:44.550525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.550489 2563 scope.go:117] "RemoveContainer" containerID="a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7" Jul 29 12:33:44.561660 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.561640 2563 scope.go:117] "RemoveContainer" containerID="a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7" Jul 29 12:33:44.562005 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:33:44.561943 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7\": container with ID starting with a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7 not found: ID does not exist" containerID="a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7" Jul 29 12:33:44.562005 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.561980 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7"} err="failed to get container status \"a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7\": rpc error: code = NotFound desc = could not find container \"a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7\": container with ID starting with a16e92661a2c251f707f7102d3956f264e4cbbe320d9c6c552b64dbbae84a4e7 not found: ID does not exist" Jul 29 12:33:44.585125 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.585092 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d"] Jul 29 12:33:44.587244 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.587219 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zkj4d"] Jul 29 12:33:44.660670 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:44.660608 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql"] Jul 29 12:33:44.663384 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:33:44.663352 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded28d431_e95b_4be8_8bfa_3770bc321292.slice/crio-a3c40c0190f51ad7c8d5a07b074cc2823693eed94de27d2ac42f683b6b9ccc28 WatchSource:0}: Error finding container a3c40c0190f51ad7c8d5a07b074cc2823693eed94de27d2ac42f683b6b9ccc28: Status 404 returned error can't find the container with id a3c40c0190f51ad7c8d5a07b074cc2823693eed94de27d2ac42f683b6b9ccc28 Jul 29 12:33:45.555897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:45.555862 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" event={"ID":"ed28d431-e95b-4be8-8bfa-3770bc321292","Type":"ContainerStarted","Data":"21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39"} Jul 29 12:33:45.556358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:45.555917 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" event={"ID":"ed28d431-e95b-4be8-8bfa-3770bc321292","Type":"ContainerStarted","Data":"a3c40c0190f51ad7c8d5a07b074cc2823693eed94de27d2ac42f683b6b9ccc28"} Jul 29 12:33:45.556358 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:45.555980 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:33:45.583245 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:45.583202 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" podStartSLOduration=1.5831879290000002 podStartE2EDuration="1.583187929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:33:45.580536822 +0000 UTC m=+498.483733499" watchObservedRunningTime="2026-07-29 12:33:45.583187929 +0000 UTC m=+498.486384607" Jul 29 12:33:45.657970 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:45.657942 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc6a63b-6aa3-45b2-bfad-07d2c4273e12" path="/var/lib/kubelet/pods/1bc6a63b-6aa3-45b2-bfad-07d2c4273e12/volumes" Jul 29 12:33:56.563930 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:33:56.563895 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:34:00.382435 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.382404 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79"] Jul 29 12:34:00.385764 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.385741 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.388704 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.388662 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kxbnm\"" Jul 29 12:34:00.402989 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.402961 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79"] Jul 29 12:34:00.416187 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.416165 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:00.482365 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482328 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aca93527-1778-4111-b825-9069d2b0fb94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aca93527-1778-4111-b825-9069d2b0fb94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482700 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482668 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aca93527-1778-4111-b825-9069d2b0fb94-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482834 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482698 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnws6\" (UniqueName: \"kubernetes.io/projected/aca93527-1778-4111-b825-9069d2b0fb94-kube-api-access-pnws6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.482834 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.482806 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.548673 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.548635 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:00.583219 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583183 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aca93527-1778-4111-b825-9069d2b0fb94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583401 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aca93527-1778-4111-b825-9069d2b0fb94-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583401 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583254 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnws6\" (UniqueName: \"kubernetes.io/projected/aca93527-1778-4111-b825-9069d2b0fb94-kube-api-access-pnws6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583401 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583401 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583323 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583401 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583681 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583681 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583443 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aca93527-1778-4111-b825-9069d2b0fb94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583838 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583813 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.583838 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583831 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.584011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583844 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.584011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.584011 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.583971 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.584160 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.584140 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aca93527-1778-4111-b825-9069d2b0fb94-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.585753 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.585733 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aca93527-1778-4111-b825-9069d2b0fb94-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.586088 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.586071 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aca93527-1778-4111-b825-9069d2b0fb94-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.591900 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.591879 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnws6\" (UniqueName: \"kubernetes.io/projected/aca93527-1778-4111-b825-9069d2b0fb94-kube-api-access-pnws6\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.592010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.591993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aca93527-1778-4111-b825-9069d2b0fb94-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-fvg79\" (UID: \"aca93527-1778-4111-b825-9069d2b0fb94\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.696179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.696094 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:00.837075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.837049 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79"] Jul 29 12:34:00.838922 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:34:00.838895 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca93527_1778_4111_b825_9069d2b0fb94.slice/crio-7b1c86817ad8de4e069f6e68d63e93af2bef4672d5df91ba5cee9f2901a0ba05 WatchSource:0}: Error finding container 7b1c86817ad8de4e069f6e68d63e93af2bef4672d5df91ba5cee9f2901a0ba05: Status 404 returned error can't find the container with id 7b1c86817ad8de4e069f6e68d63e93af2bef4672d5df91ba5cee9f2901a0ba05 Jul 29 12:34:00.840849 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.840819 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:34:00.840961 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.840880 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:34:00.840961 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:00.840912 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:34:01.627981 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:01.627947 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" event={"ID":"aca93527-1778-4111-b825-9069d2b0fb94","Type":"ContainerStarted","Data":"54fdba7634ad1d8079479a56c2aed2a0a225b38313fdfb8a41796350025f9a26"} Jul 29 12:34:01.628337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:01.628011 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" event={"ID":"aca93527-1778-4111-b825-9069d2b0fb94","Type":"ContainerStarted","Data":"7b1c86817ad8de4e069f6e68d63e93af2bef4672d5df91ba5cee9f2901a0ba05"} Jul 29 12:34:01.656767 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:01.656715 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" podStartSLOduration=1.656696228 podStartE2EDuration="1.656696228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:01.654233289 +0000 UTC m=+514.557429978" watchObservedRunningTime="2026-07-29 12:34:01.656696228 +0000 UTC m=+514.559892905" Jul 29 12:34:01.696513 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:01.696483 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:01.701081 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:01.701056 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:02.632203 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:02.632173 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:02.633210 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:02.633187 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:34:04.261099 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:04.261061 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:04.401611 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:04.401578 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:05.229041 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.229009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:05.313052 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.312988 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:05.393765 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.393739 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-2fhcd"] Jul 29 12:34:05.396971 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.396954 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:05.399943 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:34:05.399911 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-75d556bd77-2fhcd\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" podUID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:05.400062 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:34:05.399953 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-authorino-dockercfg-h26lj\" is forbidden: User \"system:node:ip-10-0-137-159.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-159.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h26lj\"" type="*v1.Secret" Jul 29 12:34:05.415400 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.415377 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2fhcd"] Jul 29 12:34:05.530777 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.530744 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpdl5\" (UniqueName: \"kubernetes.io/projected/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3-kube-api-access-hpdl5\") pod \"authorino-75d556bd77-2fhcd\" (UID: \"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3\") " pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:05.631645 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.631574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpdl5\" (UniqueName: \"kubernetes.io/projected/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3-kube-api-access-hpdl5\") pod \"authorino-75d556bd77-2fhcd\" (UID: \"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3\") " pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:05.644340 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:05.644320 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpdl5\" (UniqueName: \"kubernetes.io/projected/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3-kube-api-access-hpdl5\") pod \"authorino-75d556bd77-2fhcd\" (UID: \"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3\") " pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:06.223676 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:06.223650 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h26lj\"" Jul 29 12:34:06.226744 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:06.226714 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:06.361224 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:06.361193 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2fhcd"] Jul 29 12:34:06.364093 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:34:06.364058 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa0a5b75_7ef6_4c07_bd72_fad1efd57af3.slice/crio-1cfb736b6aa9b4900dc5893e3255467298d660779ca0985f0f0d1fc5be3fac7b WatchSource:0}: Error finding container 1cfb736b6aa9b4900dc5893e3255467298d660779ca0985f0f0d1fc5be3fac7b: Status 404 returned error can't find the container with id 1cfb736b6aa9b4900dc5893e3255467298d660779ca0985f0f0d1fc5be3fac7b Jul 29 12:34:06.650356 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:06.650285 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2fhcd" event={"ID":"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3","Type":"ContainerStarted","Data":"1cfb736b6aa9b4900dc5893e3255467298d660779ca0985f0f0d1fc5be3fac7b"} Jul 29 12:34:09.665747 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:09.665710 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2fhcd" event={"ID":"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3","Type":"ContainerStarted","Data":"8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b"} Jul 29 12:34:09.688544 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:09.688497 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-2fhcd" podStartSLOduration=1.904313785 podStartE2EDuration="4.688482274s" totalImagesPullingTime="2.784168489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:09.684586142 +0000 UTC m=+522.587782818" watchObservedRunningTime="2026-07-29 12:34:09.688482274 +0000 UTC m=+522.591678951" Jul 29 12:34:24.165248 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:24.165155 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:24.336447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:24.336416 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:24.688133 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:24.688100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:37.116989 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.116952 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:37.294420 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.294391 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:37.759298 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.759270 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6cc554ddc9-mxm2x"] Jul 29 12:34:37.763401 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.763382 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:37.765857 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.765840 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 29 12:34:37.774628 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.774604 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cc554ddc9-mxm2x"] Jul 29 12:34:37.787606 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.787586 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fa80dc0-80a0-4f3c-8332-48512c57b343-tls-cert\") pod \"authorino-6cc554ddc9-mxm2x\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:37.787689 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.787657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqw42\" (UniqueName: \"kubernetes.io/projected/1fa80dc0-80a0-4f3c-8332-48512c57b343-kube-api-access-fqw42\") pod \"authorino-6cc554ddc9-mxm2x\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:37.887953 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.887926 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fa80dc0-80a0-4f3c-8332-48512c57b343-tls-cert\") pod \"authorino-6cc554ddc9-mxm2x\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:37.888045 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.887988 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqw42\" (UniqueName: \"kubernetes.io/projected/1fa80dc0-80a0-4f3c-8332-48512c57b343-kube-api-access-fqw42\") pod \"authorino-6cc554ddc9-mxm2x\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:37.890241 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.890218 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fa80dc0-80a0-4f3c-8332-48512c57b343-tls-cert\") pod \"authorino-6cc554ddc9-mxm2x\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:37.897017 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:37.896996 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqw42\" (UniqueName: \"kubernetes.io/projected/1fa80dc0-80a0-4f3c-8332-48512c57b343-kube-api-access-fqw42\") pod \"authorino-6cc554ddc9-mxm2x\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:38.072331 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.072280 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:34:38.220767 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.220735 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6cc554ddc9-mxm2x"] Jul 29 12:34:38.224575 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:34:38.224549 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fa80dc0_80a0_4f3c_8332_48512c57b343.slice/crio-727ddab2b7746259a8d99b7f21633800e7affa63107be5ee4544bb1de298a42a WatchSource:0}: Error finding container 727ddab2b7746259a8d99b7f21633800e7affa63107be5ee4544bb1de298a42a: Status 404 returned error can't find the container with id 727ddab2b7746259a8d99b7f21633800e7affa63107be5ee4544bb1de298a42a Jul 29 12:34:38.784432 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.784389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" event={"ID":"1fa80dc0-80a0-4f3c-8332-48512c57b343","Type":"ContainerStarted","Data":"cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5"} Jul 29 12:34:38.784432 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.784432 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" event={"ID":"1fa80dc0-80a0-4f3c-8332-48512c57b343","Type":"ContainerStarted","Data":"727ddab2b7746259a8d99b7f21633800e7affa63107be5ee4544bb1de298a42a"} Jul 29 12:34:38.802903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.802853 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" podStartSLOduration=1.47012424 podStartE2EDuration="1.802839185s" totalImagesPullingTime="332.714945ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:34:38.801487965 +0000 UTC m=+551.704684640" watchObservedRunningTime="2026-07-29 12:34:38.802839185 +0000 UTC m=+551.706035878" Jul 29 12:34:38.835239 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.835172 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2fhcd"] Jul 29 12:34:38.835361 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:38.835331 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-2fhcd" podUID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" containerName="authorino" containerID="cri-o://8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b" gracePeriod=30 Jul 29 12:34:39.068833 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.068808 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:39.097627 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.097563 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpdl5\" (UniqueName: \"kubernetes.io/projected/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3-kube-api-access-hpdl5\") pod \"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3\" (UID: \"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3\") " Jul 29 12:34:39.099490 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.099431 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3-kube-api-access-hpdl5" (OuterVolumeSpecName: "kube-api-access-hpdl5") pod "fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" (UID: "fa0a5b75-7ef6-4c07-bd72-fad1efd57af3"). InnerVolumeSpecName "kube-api-access-hpdl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:34:39.198698 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.198669 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpdl5\" (UniqueName: \"kubernetes.io/projected/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3-kube-api-access-hpdl5\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:34:39.790108 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.790068 2563 generic.go:356] "Generic (PLEG): container finished" podID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" containerID="8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b" exitCode=0 Jul 29 12:34:39.790506 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.790121 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-2fhcd" Jul 29 12:34:39.790506 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.790160 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2fhcd" event={"ID":"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3","Type":"ContainerDied","Data":"8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b"} Jul 29 12:34:39.790506 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.790204 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-2fhcd" event={"ID":"fa0a5b75-7ef6-4c07-bd72-fad1efd57af3","Type":"ContainerDied","Data":"1cfb736b6aa9b4900dc5893e3255467298d660779ca0985f0f0d1fc5be3fac7b"} Jul 29 12:34:39.790506 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.790224 2563 scope.go:117] "RemoveContainer" containerID="8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b" Jul 29 12:34:39.798211 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.798194 2563 scope.go:117] "RemoveContainer" containerID="8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b" Jul 29 12:34:39.798497 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:34:39.798479 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b\": container with ID starting with 8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b not found: ID does not exist" containerID="8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b" Jul 29 12:34:39.798572 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.798504 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b"} err="failed to get container status \"8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b\": rpc error: code = NotFound desc = could not find container \"8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b\": container with ID starting with 8a867fbf04a510be25f061d4760e151f697415b1071475ea15a20f7b0a22450b not found: ID does not exist" Jul 29 12:34:39.810914 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.810889 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2fhcd"] Jul 29 12:34:39.818685 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:39.818663 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-2fhcd"] Jul 29 12:34:41.658006 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:41.657964 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" path="/var/lib/kubelet/pods/fa0a5b75-7ef6-4c07-bd72-fad1efd57af3/volumes" Jul 29 12:34:43.804754 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:43.804722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:43.897377 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:43.897338 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:43.976758 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:43.976721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:44.261875 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:44.261834 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:47.331709 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:47.331673 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:47.429045 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:47.429010 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:48.712616 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:48.712587 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:34:48.815411 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:34:48.815385 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:17.715670 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:17.715634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:18.688148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:18.688111 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:19.139895 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:19.139850 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:19.255199 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:19.255172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:19.905312 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:19.905279 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:20.220642 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.220608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:20.669549 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.669442 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:20.765630 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.765591 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:20.892473 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.892425 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh"] Jul 29 12:35:20.892816 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.892801 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" containerName="authorino" Jul 29 12:35:20.892858 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.892818 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" containerName="authorino" Jul 29 12:35:20.892895 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.892881 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa0a5b75-7ef6-4c07-bd72-fad1efd57af3" containerName="authorino" Jul 29 12:35:20.895684 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.895668 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:20.898706 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.898681 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kwmrt\"" Jul 29 12:35:20.898833 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.898692 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 29 12:35:20.905788 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:20.905765 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh"] Jul 29 12:35:21.026975 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.026947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e873a36-bb9e-4e02-aaf6-8c19bc718d1d-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-2prvh\" (UID: \"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.027140 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.026990 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gd6d\" (UniqueName: \"kubernetes.io/projected/6e873a36-bb9e-4e02-aaf6-8c19bc718d1d-kube-api-access-8gd6d\") pod \"payload-pre-processing-687dbb6cf6-2prvh\" (UID: \"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.127585 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.127552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e873a36-bb9e-4e02-aaf6-8c19bc718d1d-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-2prvh\" (UID: \"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.127732 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.127596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gd6d\" (UniqueName: \"kubernetes.io/projected/6e873a36-bb9e-4e02-aaf6-8c19bc718d1d-kube-api-access-8gd6d\") pod \"payload-pre-processing-687dbb6cf6-2prvh\" (UID: \"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.128218 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.128199 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e873a36-bb9e-4e02-aaf6-8c19bc718d1d-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-2prvh\" (UID: \"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.141684 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.141656 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gd6d\" (UniqueName: \"kubernetes.io/projected/6e873a36-bb9e-4e02-aaf6-8c19bc718d1d-kube-api-access-8gd6d\") pod \"payload-pre-processing-687dbb6cf6-2prvh\" (UID: \"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.205819 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.205792 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:21.344600 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.344574 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh"] Jul 29 12:35:21.346622 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:35:21.346593 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e873a36_bb9e_4e02_aaf6_8c19bc718d1d.slice/crio-6efdcab8955f5683a9ca716f42fe007dd123f02beafbcdfea6f61b851d6f6eaa WatchSource:0}: Error finding container 6efdcab8955f5683a9ca716f42fe007dd123f02beafbcdfea6f61b851d6f6eaa: Status 404 returned error can't find the container with id 6efdcab8955f5683a9ca716f42fe007dd123f02beafbcdfea6f61b851d6f6eaa Jul 29 12:35:21.990819 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:21.990779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" event={"ID":"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d","Type":"ContainerStarted","Data":"6efdcab8955f5683a9ca716f42fe007dd123f02beafbcdfea6f61b851d6f6eaa"} Jul 29 12:35:25.007035 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:25.006993 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" event={"ID":"6e873a36-bb9e-4e02-aaf6-8c19bc718d1d","Type":"ContainerStarted","Data":"d75fd0e6fe1f503d6fffbdfee383d9c43b4739838181c0acf1d5637ef30322e4"} Jul 29 12:35:25.007422 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:25.007099 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:25.024530 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:25.024477 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" podStartSLOduration=2.251476908 podStartE2EDuration="5.024441533s" totalImagesPullingTime="2.772964625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:35:25.024193691 +0000 UTC m=+597.927390369" watchObservedRunningTime="2026-07-29 12:35:25.024441533 +0000 UTC m=+597.927638210" Jul 29 12:35:29.025221 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:29.025192 2563 generic.go:356] "Generic (PLEG): container finished" podID="aca93527-1778-4111-b825-9069d2b0fb94" containerID="54fdba7634ad1d8079479a56c2aed2a0a225b38313fdfb8a41796350025f9a26" exitCode=0 Jul 29 12:35:29.025616 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:29.025247 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" event={"ID":"aca93527-1778-4111-b825-9069d2b0fb94","Type":"ContainerDied","Data":"54fdba7634ad1d8079479a56c2aed2a0a225b38313fdfb8a41796350025f9a26"} Jul 29 12:35:29.025616 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:29.025545 2563 scope.go:117] "RemoveContainer" containerID="54fdba7634ad1d8079479a56c2aed2a0a225b38313fdfb8a41796350025f9a26" Jul 29 12:35:29.026371 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:29.026344 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:35:29.026424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:29.026410 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:35:29.026475 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:29.026442 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:35:30.031311 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:30.031280 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" event={"ID":"aca93527-1778-4111-b825-9069d2b0fb94","Type":"ContainerStarted","Data":"72ede37bc32ec552151eec63dba9f56d0cffbb5ca0dc033a6eefefeed9f53898"} Jul 29 12:35:30.696971 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:30.696935 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:35:30.697956 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:30.697933 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fvg79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 29 12:35:30.698019 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:30.697978 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" podUID="aca93527-1778-4111-b825-9069d2b0fb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 29 12:35:30.854315 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:30.854283 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:31.696994 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:31.696956 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fvg79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 29 12:35:31.697377 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:31.697006 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" podUID="aca93527-1778-4111-b825-9069d2b0fb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 29 12:35:32.399066 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:32.399030 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:32.632857 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:32.632821 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:35:32.697569 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:32.697450 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-fvg79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body= Jul 29 12:35:32.697569 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:32.697550 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" podUID="aca93527-1778-4111-b825-9069d2b0fb94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" Jul 29 12:35:33.730952 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:33.730920 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:35:33.731933 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:33.731913 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-fvg79" Jul 29 12:35:36.013776 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:36.013744 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-2prvh" Jul 29 12:35:38.110723 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:38.110691 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:50.869647 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.869574 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj"] Jul 29 12:35:50.876623 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.876604 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:50.879075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.879054 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 29 12:35:50.879944 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.879920 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 29 12:35:50.880016 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.879974 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mh475\"" Jul 29 12:35:50.880066 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.879974 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 29 12:35:50.885780 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:50.885760 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj"] Jul 29 12:35:51.063279 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.063239 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv4mn\" (UniqueName: \"kubernetes.io/projected/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-kube-api-access-tv4mn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.063279 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.063277 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.063525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.063317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.063525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.063401 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.063525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.063419 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.063525 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.063439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.163878 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.163803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.163878 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.163848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.163878 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.163862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.164368 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.163880 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.164368 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.163912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv4mn\" (UniqueName: \"kubernetes.io/projected/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-kube-api-access-tv4mn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.164368 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.163940 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.164368 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.164331 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.164564 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.164385 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.164564 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.164490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.166057 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.166036 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.166732 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.166712 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.172812 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.172788 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv4mn\" (UniqueName: \"kubernetes.io/projected/ac66994e-ce1d-4ed1-97e0-4393e0ab1b15-kube-api-access-tv4mn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mp7hj\" (UID: \"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.186839 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.186818 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:35:51.240355 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.240323 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:51.315822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.315772 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj"] Jul 29 12:35:51.318143 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:35:51.318106 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac66994e_ce1d_4ed1_97e0_4393e0ab1b15.slice/crio-0f92612a3684f8c0a3b7fc174999deec25620a2585a2ee4a8cb7ecd523caa4ad WatchSource:0}: Error finding container 0f92612a3684f8c0a3b7fc174999deec25620a2585a2ee4a8cb7ecd523caa4ad: Status 404 returned error can't find the container with id 0f92612a3684f8c0a3b7fc174999deec25620a2585a2ee4a8cb7ecd523caa4ad Jul 29 12:35:51.319816 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:51.319797 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 12:35:52.133514 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:52.133448 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" event={"ID":"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15","Type":"ContainerStarted","Data":"0f92612a3684f8c0a3b7fc174999deec25620a2585a2ee4a8cb7ecd523caa4ad"} Jul 29 12:35:53.141361 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:53.141324 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" event={"ID":"ac66994e-ce1d-4ed1-97e0-4393e0ab1b15","Type":"ContainerStarted","Data":"37b640f6c7ccb761175f433f7417cc3acf3ed66b19c2c1b2ded4ad3b23627477"} Jul 29 12:35:53.162794 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:53.162735 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" podStartSLOduration=2.102228847 podStartE2EDuration="3.16271712s" totalImagesPullingTime="1.060488273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:35:53.16190317 +0000 UTC m=+626.065099847" watchObservedRunningTime="2026-07-29 12:35:53.16271712 +0000 UTC m=+626.065913796" Jul 29 12:35:59.137019 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:59.136987 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:35:59.238369 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:35:59.238340 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:01.187513 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:01.187485 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:36:01.187875 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:01.187635 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:36:01.199899 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:01.199878 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:36:02.190991 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:02.190964 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mp7hj" Jul 29 12:36:03.020292 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:03.020257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:07.425083 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:07.425046 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:08.142923 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:08.142888 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:08.247085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:08.247048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:16.558590 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.558560 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk"] Jul 29 12:36:16.562135 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.562113 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.564655 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.564633 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 29 12:36:16.583771 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.583746 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk"] Jul 29 12:36:16.605859 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.605838 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:16.664215 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.664189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.664315 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.664222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.664315 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.664240 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8sc\" (UniqueName: \"kubernetes.io/projected/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-kube-api-access-bw8sc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.664424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.664325 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.664424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.664355 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.664424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.664371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.765694 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.765658 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.765694 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.765698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.765897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.765719 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8sc\" (UniqueName: \"kubernetes.io/projected/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-kube-api-access-bw8sc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.765897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.765769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.765897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.765789 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.765897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.765803 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.766148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.766124 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.766205 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.766159 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.766254 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.766202 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.768021 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.767997 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.768429 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.768407 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.775807 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.775787 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8sc\" (UniqueName: \"kubernetes.io/projected/4cad38d4-d1db-4cdd-8d0c-a8a4005adc64-kube-api-access-bw8sc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4jkbk\" (UID: \"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.871491 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.871418 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:16.996439 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:16.996415 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk"] Jul 29 12:36:16.998418 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:36:16.998390 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cad38d4_d1db_4cdd_8d0c_a8a4005adc64.slice/crio-c488a9d7fa62e9fc55da7114f87438d019feee53a0d1a68c075eb485f5f29be8 WatchSource:0}: Error finding container c488a9d7fa62e9fc55da7114f87438d019feee53a0d1a68c075eb485f5f29be8: Status 404 returned error can't find the container with id c488a9d7fa62e9fc55da7114f87438d019feee53a0d1a68c075eb485f5f29be8 Jul 29 12:36:17.246634 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:17.246610 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" event={"ID":"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64","Type":"ContainerStarted","Data":"c488a9d7fa62e9fc55da7114f87438d019feee53a0d1a68c075eb485f5f29be8"} Jul 29 12:36:18.252552 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:18.252516 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" event={"ID":"4cad38d4-d1db-4cdd-8d0c-a8a4005adc64","Type":"ContainerStarted","Data":"aafad253c9d0c2d73e3425060feced7e0d8708417720b386e4864c21868a18f0"} Jul 29 12:36:18.277743 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:18.277698 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" podStartSLOduration=2.082592805 podStartE2EDuration="2.277684197s" totalImagesPullingTime="195.091392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:36:18.2760289 +0000 UTC m=+651.179225575" watchObservedRunningTime="2026-07-29 12:36:18.277684197 +0000 UTC m=+651.180880873" Jul 29 12:36:20.539927 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:20.539884 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:36:26.871601 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:26.871564 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:26.871973 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:26.871632 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:26.884292 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:26.884267 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:36:27.301728 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:36:27.301700 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4jkbk" Jul 29 12:37:11.989239 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:11.989197 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:37:12.132937 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:12.132907 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:37:12.908515 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:12.908482 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cc554ddc9-mxm2x"] Jul 29 12:37:12.908694 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:12.908658 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" podUID="1fa80dc0-80a0-4f3c-8332-48512c57b343" containerName="authorino" containerID="cri-o://cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5" gracePeriod=30 Jul 29 12:37:13.159755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.159701 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:37:13.193892 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.193868 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqw42\" (UniqueName: \"kubernetes.io/projected/1fa80dc0-80a0-4f3c-8332-48512c57b343-kube-api-access-fqw42\") pod \"1fa80dc0-80a0-4f3c-8332-48512c57b343\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " Jul 29 12:37:13.194030 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.193971 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fa80dc0-80a0-4f3c-8332-48512c57b343-tls-cert\") pod \"1fa80dc0-80a0-4f3c-8332-48512c57b343\" (UID: \"1fa80dc0-80a0-4f3c-8332-48512c57b343\") " Jul 29 12:37:13.195982 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.195953 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fa80dc0-80a0-4f3c-8332-48512c57b343-kube-api-access-fqw42" (OuterVolumeSpecName: "kube-api-access-fqw42") pod "1fa80dc0-80a0-4f3c-8332-48512c57b343" (UID: "1fa80dc0-80a0-4f3c-8332-48512c57b343"). InnerVolumeSpecName "kube-api-access-fqw42". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:37:13.205399 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.205370 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fa80dc0-80a0-4f3c-8332-48512c57b343-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1fa80dc0-80a0-4f3c-8332-48512c57b343" (UID: "1fa80dc0-80a0-4f3c-8332-48512c57b343"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 12:37:13.252065 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.252041 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:37:13.295535 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.295511 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1fa80dc0-80a0-4f3c-8332-48512c57b343-tls-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:37:13.295535 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.295533 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqw42\" (UniqueName: \"kubernetes.io/projected/1fa80dc0-80a0-4f3c-8332-48512c57b343-kube-api-access-fqw42\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:37:13.348854 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.348832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:37:13.485086 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.485058 2563 generic.go:356] "Generic (PLEG): container finished" podID="1fa80dc0-80a0-4f3c-8332-48512c57b343" containerID="cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5" exitCode=0 Jul 29 12:37:13.485213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.485100 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" Jul 29 12:37:13.485213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.485146 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" event={"ID":"1fa80dc0-80a0-4f3c-8332-48512c57b343","Type":"ContainerDied","Data":"cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5"} Jul 29 12:37:13.485213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.485184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6cc554ddc9-mxm2x" event={"ID":"1fa80dc0-80a0-4f3c-8332-48512c57b343","Type":"ContainerDied","Data":"727ddab2b7746259a8d99b7f21633800e7affa63107be5ee4544bb1de298a42a"} Jul 29 12:37:13.485213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.485201 2563 scope.go:117] "RemoveContainer" containerID="cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5" Jul 29 12:37:13.493755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.493740 2563 scope.go:117] "RemoveContainer" containerID="cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5" Jul 29 12:37:13.493996 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:37:13.493980 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5\": container with ID starting with cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5 not found: ID does not exist" containerID="cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5" Jul 29 12:37:13.494051 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.494006 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5"} err="failed to get container status \"cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5\": rpc error: code = NotFound desc = could not find container \"cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5\": container with ID starting with cbda83cb98550b0c4f2ce1d1043cdac615e4a37c79712a589779ec7274b0ebf5 not found: ID does not exist" Jul 29 12:37:13.528402 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.528375 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6cc554ddc9-mxm2x"] Jul 29 12:37:13.529950 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.529931 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6cc554ddc9-mxm2x"] Jul 29 12:37:13.657751 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:37:13.657722 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa80dc0-80a0-4f3c-8332-48512c57b343" path="/var/lib/kubelet/pods/1fa80dc0-80a0-4f3c-8332-48512c57b343/volumes" Jul 29 12:38:06.524238 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:38:06.524204 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:38:06.607654 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:38:06.607623 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:39:04.117016 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:04.116981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:39:19.043115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:19.043085 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:39:20.478229 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.478192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:39:20.589124 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.589087 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65d577d9d5-86zjt"] Jul 29 12:39:20.589471 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.589441 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa80dc0-80a0-4f3c-8332-48512c57b343" containerName="authorino" Jul 29 12:39:20.589561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.589473 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa80dc0-80a0-4f3c-8332-48512c57b343" containerName="authorino" Jul 29 12:39:20.589618 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.589583 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa80dc0-80a0-4f3c-8332-48512c57b343" containerName="authorino" Jul 29 12:39:20.592451 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.592431 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.596399 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.596375 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 29 12:39:20.596536 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.596378 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 29 12:39:20.596536 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.596378 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kjvkj\"" Jul 29 12:39:20.603758 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.603737 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-86zjt"] Jul 29 12:39:20.663141 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.663118 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8657e2a-3c9c-4b07-a031-1d0a6ae4424c-webhook-cert\") pod \"maas-controller-65d577d9d5-86zjt\" (UID: \"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c\") " pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.663260 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.663204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45czg\" (UniqueName: \"kubernetes.io/projected/c8657e2a-3c9c-4b07-a031-1d0a6ae4424c-kube-api-access-45czg\") pod \"maas-controller-65d577d9d5-86zjt\" (UID: \"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c\") " pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.764176 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.764127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8657e2a-3c9c-4b07-a031-1d0a6ae4424c-webhook-cert\") pod \"maas-controller-65d577d9d5-86zjt\" (UID: \"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c\") " pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.764271 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.764185 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45czg\" (UniqueName: \"kubernetes.io/projected/c8657e2a-3c9c-4b07-a031-1d0a6ae4424c-kube-api-access-45czg\") pod \"maas-controller-65d577d9d5-86zjt\" (UID: \"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c\") " pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.766519 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.766500 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8657e2a-3c9c-4b07-a031-1d0a6ae4424c-webhook-cert\") pod \"maas-controller-65d577d9d5-86zjt\" (UID: \"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c\") " pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.774108 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.774083 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45czg\" (UniqueName: \"kubernetes.io/projected/c8657e2a-3c9c-4b07-a031-1d0a6ae4424c-kube-api-access-45czg\") pod \"maas-controller-65d577d9d5-86zjt\" (UID: \"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c\") " pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:20.903430 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:20.903408 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:21.032253 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:21.032184 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65d577d9d5-86zjt"] Jul 29 12:39:21.034507 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:39:21.034481 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8657e2a_3c9c_4b07_a031_1d0a6ae4424c.slice/crio-55665dfca9990702932175c5fd51d3c15a214d8242d2d985af95881c237dc1d4 WatchSource:0}: Error finding container 55665dfca9990702932175c5fd51d3c15a214d8242d2d985af95881c237dc1d4: Status 404 returned error can't find the container with id 55665dfca9990702932175c5fd51d3c15a214d8242d2d985af95881c237dc1d4 Jul 29 12:39:22.035861 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:22.035825 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-86zjt" event={"ID":"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c","Type":"ContainerStarted","Data":"55665dfca9990702932175c5fd51d3c15a214d8242d2d985af95881c237dc1d4"} Jul 29 12:39:24.047216 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:24.047178 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65d577d9d5-86zjt" event={"ID":"c8657e2a-3c9c-4b07-a031-1d0a6ae4424c","Type":"ContainerStarted","Data":"2620e3b2d5f4926b6844e87889135a8b7236dc070990aa06dbe2cf2fbefa3a33"} Jul 29 12:39:24.047584 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:24.047300 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:39:24.072241 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:24.072199 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65d577d9d5-86zjt" podStartSLOduration=1.625527242 podStartE2EDuration="4.072184502s" totalImagesPullingTime="2.44665726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:39:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:39:24.071835503 +0000 UTC m=+836.975032180" watchObservedRunningTime="2026-07-29 12:39:24.072184502 +0000 UTC m=+836.975381177" Jul 29 12:39:35.053816 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:39:35.053787 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65d577d9d5-86zjt" Jul 29 12:40:03.176746 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:03.176708 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:30.543628 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:30.543568 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:30.625087 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:30.625059 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:30.776726 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:30.776692 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:36.924169 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:36.924137 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:39.181572 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:39.181532 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:43.437439 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:43.437405 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:40:43.516864 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:40:43.516825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:44:58.877782 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:44:58.877746 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:45:19.291265 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:45:19.291230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:45:27.762028 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:45:27.761993 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:46:30.043351 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.043280 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql"] Jul 29 12:46:30.045907 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.043473 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" podUID="ed28d431-e95b-4be8-8bfa-3770bc321292" containerName="manager" containerID="cri-o://21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39" gracePeriod=10 Jul 29 12:46:30.307675 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.307652 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:46:30.338715 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.338688 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed28d431-e95b-4be8-8bfa-3770bc321292-extensions-socket-volume\") pod \"ed28d431-e95b-4be8-8bfa-3770bc321292\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " Jul 29 12:46:30.338858 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.338745 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw264\" (UniqueName: \"kubernetes.io/projected/ed28d431-e95b-4be8-8bfa-3770bc321292-kube-api-access-rw264\") pod \"ed28d431-e95b-4be8-8bfa-3770bc321292\" (UID: \"ed28d431-e95b-4be8-8bfa-3770bc321292\") " Jul 29 12:46:30.339075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.339049 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed28d431-e95b-4be8-8bfa-3770bc321292-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ed28d431-e95b-4be8-8bfa-3770bc321292" (UID: "ed28d431-e95b-4be8-8bfa-3770bc321292"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:46:30.340602 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.340580 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed28d431-e95b-4be8-8bfa-3770bc321292-kube-api-access-rw264" (OuterVolumeSpecName: "kube-api-access-rw264") pod "ed28d431-e95b-4be8-8bfa-3770bc321292" (UID: "ed28d431-e95b-4be8-8bfa-3770bc321292"). InnerVolumeSpecName "kube-api-access-rw264". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:46:30.439217 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.439190 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed28d431-e95b-4be8-8bfa-3770bc321292-extensions-socket-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:46:30.439217 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.439214 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw264\" (UniqueName: \"kubernetes.io/projected/ed28d431-e95b-4be8-8bfa-3770bc321292-kube-api-access-rw264\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:46:30.889963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.889927 2563 generic.go:356] "Generic (PLEG): container finished" podID="ed28d431-e95b-4be8-8bfa-3770bc321292" containerID="21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39" exitCode=0 Jul 29 12:46:30.890106 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.889971 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" Jul 29 12:46:30.890106 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.890007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" event={"ID":"ed28d431-e95b-4be8-8bfa-3770bc321292","Type":"ContainerDied","Data":"21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39"} Jul 29 12:46:30.890106 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.890056 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql" event={"ID":"ed28d431-e95b-4be8-8bfa-3770bc321292","Type":"ContainerDied","Data":"a3c40c0190f51ad7c8d5a07b074cc2823693eed94de27d2ac42f683b6b9ccc28"} Jul 29 12:46:30.890106 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.890079 2563 scope.go:117] "RemoveContainer" containerID="21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39" Jul 29 12:46:30.899434 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.899415 2563 scope.go:117] "RemoveContainer" containerID="21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39" Jul 29 12:46:30.899711 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:46:30.899692 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39\": container with ID starting with 21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39 not found: ID does not exist" containerID="21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39" Jul 29 12:46:30.899756 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.899720 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39"} err="failed to get container status \"21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39\": rpc error: code = NotFound desc = could not find container \"21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39\": container with ID starting with 21cf7514e3332f6d40c0eee8581ae406a88b1f294e1b0009b06fc6fa1d0c2e39 not found: ID does not exist" Jul 29 12:46:30.911983 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.911963 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql"] Jul 29 12:46:30.916367 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:30.916346 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-48bql"] Jul 29 12:46:31.658617 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:46:31.658586 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed28d431-e95b-4be8-8bfa-3770bc321292" path="/var/lib/kubelet/pods/ed28d431-e95b-4be8-8bfa-3770bc321292/volumes" Jul 29 12:47:30.806471 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.806426 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5"] Jul 29 12:47:30.806849 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.806782 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed28d431-e95b-4be8-8bfa-3770bc321292" containerName="manager" Jul 29 12:47:30.806849 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.806796 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed28d431-e95b-4be8-8bfa-3770bc321292" containerName="manager" Jul 29 12:47:30.806849 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.806847 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed28d431-e95b-4be8-8bfa-3770bc321292" containerName="manager" Jul 29 12:47:30.810076 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.810060 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:30.816198 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.815875 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9h9ms\"" Jul 29 12:47:30.827772 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:30.827746 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5"] Jul 29 12:47:31.005116 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.005078 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv2pc\" (UniqueName: \"kubernetes.io/projected/31fa3226-21d5-4abc-a89d-c403df04efe7-kube-api-access-nv2pc\") pod \"kuadrant-operator-controller-manager-86878c769f-h8rj5\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.005293 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.005197 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31fa3226-21d5-4abc-a89d-c403df04efe7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8rj5\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.106451 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.106373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv2pc\" (UniqueName: \"kubernetes.io/projected/31fa3226-21d5-4abc-a89d-c403df04efe7-kube-api-access-nv2pc\") pod \"kuadrant-operator-controller-manager-86878c769f-h8rj5\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.106619 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.106456 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31fa3226-21d5-4abc-a89d-c403df04efe7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8rj5\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.106803 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.106784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31fa3226-21d5-4abc-a89d-c403df04efe7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h8rj5\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.115711 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.115688 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv2pc\" (UniqueName: \"kubernetes.io/projected/31fa3226-21d5-4abc-a89d-c403df04efe7-kube-api-access-nv2pc\") pod \"kuadrant-operator-controller-manager-86878c769f-h8rj5\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.119505 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.119479 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:31.254718 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:31.254694 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5"] Jul 29 12:47:31.257773 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:47:31.257742 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31fa3226_21d5_4abc_a89d_c403df04efe7.slice/crio-0260b5f00ca60b3b01deb6e48f012e193fa2f36c86f0704a0769ef66fa50362c WatchSource:0}: Error finding container 0260b5f00ca60b3b01deb6e48f012e193fa2f36c86f0704a0769ef66fa50362c: Status 404 returned error can't find the container with id 0260b5f00ca60b3b01deb6e48f012e193fa2f36c86f0704a0769ef66fa50362c Jul 29 12:47:32.161779 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:32.161742 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" event={"ID":"31fa3226-21d5-4abc-a89d-c403df04efe7","Type":"ContainerStarted","Data":"e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335"} Jul 29 12:47:32.162183 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:32.161869 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" event={"ID":"31fa3226-21d5-4abc-a89d-c403df04efe7","Type":"ContainerStarted","Data":"0260b5f00ca60b3b01deb6e48f012e193fa2f36c86f0704a0769ef66fa50362c"} Jul 29 12:47:32.162183 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:32.161906 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:47:32.183329 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:32.183279 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" podStartSLOduration=2.183265129 podStartE2EDuration="2.183265129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:47:32.181760511 +0000 UTC m=+1325.084957188" watchObservedRunningTime="2026-07-29 12:47:32.183265129 +0000 UTC m=+1325.086461804" Jul 29 12:47:43.168450 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:47:43.168419 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 12:48:43.550184 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:48:43.550147 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:51:09.689480 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:09.689420 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:51:15.454327 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.454293 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf"] Jul 29 12:51:15.457873 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.457853 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.460483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.460441 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f1d16833-kserve-self-signed-certs\"" Jul 29 12:51:15.467844 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.467823 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf"] Jul 29 12:51:15.488209 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.488188 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:51:15.552309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.552280 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-model-cache\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.552442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.552316 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-dshm\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.552442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.552347 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-tmp-dir\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.552442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.552386 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c807ecb-d84e-4756-a1f9-413849d6a340-tls-certs\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.552442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.552406 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-home\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.552442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.552430 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svlsw\" (UniqueName: \"kubernetes.io/projected/4c807ecb-d84e-4756-a1f9-413849d6a340-kube-api-access-svlsw\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.652966 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.652927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c807ecb-d84e-4756-a1f9-413849d6a340-tls-certs\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.652966 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.652969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-home\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svlsw\" (UniqueName: \"kubernetes.io/projected/4c807ecb-d84e-4756-a1f9-413849d6a340-kube-api-access-svlsw\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-model-cache\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-dshm\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-tmp-dir\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653447 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-home\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653564 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653499 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-model-cache\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.653564 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.653539 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-tmp-dir\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.655341 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.655316 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-dshm\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.655442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.655373 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c807ecb-d84e-4756-a1f9-413849d6a340-tls-certs\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.662207 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.661796 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svlsw\" (UniqueName: \"kubernetes.io/projected/4c807ecb-d84e-4756-a1f9-413849d6a340-kube-api-access-svlsw\") pod \"e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.767592 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.767554 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:15.896760 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:15.896731 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf"] Jul 29 12:51:15.897965 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:51:15.897940 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c807ecb_d84e_4756_a1f9_413849d6a340.slice/crio-f1e44e47e50065b55e9f1e152dd2df20dec6a19a1183389b0b62f2d3161dc40f WatchSource:0}: Error finding container f1e44e47e50065b55e9f1e152dd2df20dec6a19a1183389b0b62f2d3161dc40f: Status 404 returned error can't find the container with id f1e44e47e50065b55e9f1e152dd2df20dec6a19a1183389b0b62f2d3161dc40f Jul 29 12:51:16.108291 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:16.108214 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" event={"ID":"4c807ecb-d84e-4756-a1f9-413849d6a340","Type":"ContainerStarted","Data":"9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352"} Jul 29 12:51:16.108291 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:16.108255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" event={"ID":"4c807ecb-d84e-4756-a1f9-413849d6a340","Type":"ContainerStarted","Data":"f1e44e47e50065b55e9f1e152dd2df20dec6a19a1183389b0b62f2d3161dc40f"} Jul 29 12:51:16.129001 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:16.128958 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" podStartSLOduration=1.128942826 podStartE2EDuration="1.128942826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:51:16.126722368 +0000 UTC m=+1549.029919044" watchObservedRunningTime="2026-07-29 12:51:16.128942826 +0000 UTC m=+1549.032139502" Jul 29 12:51:25.768021 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:25.767987 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:25.768021 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:25.768029 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:25.780404 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:25.780379 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:26.162486 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:26.162405 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:51:44.876963 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:44.876918 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:51:45.466424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:45.466391 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf"] Jul 29 12:51:45.488052 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:51:45.488018 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:52:00.485126 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.485083 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" podUID="4c807ecb-d84e-4756-a1f9-413849d6a340" containerName="main" containerID="cri-o://9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352" gracePeriod=45 Jul 29 12:52:00.721947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.721923 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:52:00.808269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808205 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-model-cache\") pod \"4c807ecb-d84e-4756-a1f9-413849d6a340\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " Jul 29 12:52:00.808269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808240 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-tmp-dir\") pod \"4c807ecb-d84e-4756-a1f9-413849d6a340\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " Jul 29 12:52:00.808269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808262 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-home\") pod \"4c807ecb-d84e-4756-a1f9-413849d6a340\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " Jul 29 12:52:00.808495 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808295 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svlsw\" (UniqueName: \"kubernetes.io/projected/4c807ecb-d84e-4756-a1f9-413849d6a340-kube-api-access-svlsw\") pod \"4c807ecb-d84e-4756-a1f9-413849d6a340\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " Jul 29 12:52:00.808495 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808321 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-dshm\") pod \"4c807ecb-d84e-4756-a1f9-413849d6a340\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " Jul 29 12:52:00.808495 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808367 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c807ecb-d84e-4756-a1f9-413849d6a340-tls-certs\") pod \"4c807ecb-d84e-4756-a1f9-413849d6a340\" (UID: \"4c807ecb-d84e-4756-a1f9-413849d6a340\") " Jul 29 12:52:00.808654 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808487 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c807ecb-d84e-4756-a1f9-413849d6a340" (UID: "4c807ecb-d84e-4756-a1f9-413849d6a340"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:52:00.808654 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808523 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c807ecb-d84e-4756-a1f9-413849d6a340" (UID: "4c807ecb-d84e-4756-a1f9-413849d6a340"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:52:00.808654 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808546 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-home" (OuterVolumeSpecName: "home") pod "4c807ecb-d84e-4756-a1f9-413849d6a340" (UID: "4c807ecb-d84e-4756-a1f9-413849d6a340"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:52:00.808790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808678 2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-model-cache\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:52:00.808790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808689 2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-tmp-dir\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:52:00.808790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.808700 2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-home\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:52:00.810539 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.810517 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-dshm" (OuterVolumeSpecName: "dshm") pod "4c807ecb-d84e-4756-a1f9-413849d6a340" (UID: "4c807ecb-d84e-4756-a1f9-413849d6a340"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:52:00.810621 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.810538 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c807ecb-d84e-4756-a1f9-413849d6a340-kube-api-access-svlsw" (OuterVolumeSpecName: "kube-api-access-svlsw") pod "4c807ecb-d84e-4756-a1f9-413849d6a340" (UID: "4c807ecb-d84e-4756-a1f9-413849d6a340"). InnerVolumeSpecName "kube-api-access-svlsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:52:00.810621 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.810597 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c807ecb-d84e-4756-a1f9-413849d6a340-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c807ecb-d84e-4756-a1f9-413849d6a340" (UID: "4c807ecb-d84e-4756-a1f9-413849d6a340"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 12:52:00.909803 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.909774 2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c807ecb-d84e-4756-a1f9-413849d6a340-tls-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:52:00.909803 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.909800 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svlsw\" (UniqueName: \"kubernetes.io/projected/4c807ecb-d84e-4756-a1f9-413849d6a340-kube-api-access-svlsw\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:52:00.909952 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:00.909813 2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c807ecb-d84e-4756-a1f9-413849d6a340-dshm\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:52:01.311980 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.311942 2563 generic.go:356] "Generic (PLEG): container finished" podID="4c807ecb-d84e-4756-a1f9-413849d6a340" containerID="9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352" exitCode=0 Jul 29 12:52:01.312140 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.312005 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" Jul 29 12:52:01.312140 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.312016 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" event={"ID":"4c807ecb-d84e-4756-a1f9-413849d6a340","Type":"ContainerDied","Data":"9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352"} Jul 29 12:52:01.312140 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.312064 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf" event={"ID":"4c807ecb-d84e-4756-a1f9-413849d6a340","Type":"ContainerDied","Data":"f1e44e47e50065b55e9f1e152dd2df20dec6a19a1183389b0b62f2d3161dc40f"} Jul 29 12:52:01.312140 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.312082 2563 scope.go:117] "RemoveContainer" containerID="9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352" Jul 29 12:52:01.320278 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.320262 2563 scope.go:117] "RemoveContainer" containerID="9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352" Jul 29 12:52:01.320643 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:52:01.320622 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352\": container with ID starting with 9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352 not found: ID does not exist" containerID="9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352" Jul 29 12:52:01.320703 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.320651 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352"} err="failed to get container status \"9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352\": rpc error: code = NotFound desc = could not find container \"9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352\": container with ID starting with 9efad45afe38558a20540a22d0ce81f14d82064e79f4bc6eeed129e962bcc352 not found: ID does not exist" Jul 29 12:52:01.332235 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.332215 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf"] Jul 29 12:52:01.335559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.335538 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f1d16833-kserve-9fb66f4b7-86nhf"] Jul 29 12:52:01.658645 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:52:01.658575 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c807ecb-d84e-4756-a1f9-413849d6a340" path="/var/lib/kubelet/pods/4c807ecb-d84e-4756-a1f9-413849d6a340/volumes" Jul 29 12:54:10.211790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:54:10.211759 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:54:15.984422 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:54:15.984386 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:55:49.815396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:55:49.815358 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:26.398016 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:26.397980 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:29.505712 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:29.505668 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:29.603854 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:29.603828 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:29.966512 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:29.966476 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:35.612435 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:35.612398 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:35.711860 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:35.711831 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:35.847387 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:35.847350 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:36.136119 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.136090 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj"] Jul 29 12:56:36.136448 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.136433 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c807ecb-d84e-4756-a1f9-413849d6a340" containerName="main" Jul 29 12:56:36.136448 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.136449 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c807ecb-d84e-4756-a1f9-413849d6a340" containerName="main" Jul 29 12:56:36.136560 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.136518 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c807ecb-d84e-4756-a1f9-413849d6a340" containerName="main" Jul 29 12:56:36.139320 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.139302 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.141678 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.141660 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3cf3b144-openshift-default-dockercfg-dnpvc\"" Jul 29 12:56:36.165347 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.165329 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj"] Jul 29 12:56:36.165898 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.165858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:36.237373 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-podinfo\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-envoy\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istiod-ca-cert\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237501 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-credential-socket\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-certs\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-data\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-token\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237691 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-socket\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.237811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.237718 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq2bt\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-kube-api-access-lq2bt\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338314 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338287 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq2bt\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-kube-api-access-lq2bt\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338314 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-podinfo\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338340 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-envoy\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338360 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istiod-ca-cert\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-credential-socket\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338417 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-certs\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338437 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-data\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338573 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338499 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-token\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338870 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-socket\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338870 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338835 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-certs\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338976 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338882 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-credential-socket\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.338976 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.338919 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-socket\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.339099 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.339084 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-data\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.339300 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.339284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istiod-ca-cert\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.340983 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.340956 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-envoy\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.341188 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.341170 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-podinfo\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.346906 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.346884 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-token\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.347313 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.347292 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq2bt\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-kube-api-access-lq2bt\") pod \"e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.450626 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.450569 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:36.585289 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.585223 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj"] Jul 29 12:56:36.588245 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:56:36.588188 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d1f840_a06b_4eb4_82ef_bd8538f83e7a.slice/crio-70c767ebebfef417bc1f70aadd3756ce32542592c1c569a46c406bee18376b2b WatchSource:0}: Error finding container 70c767ebebfef417bc1f70aadd3756ce32542592c1c569a46c406bee18376b2b: Status 404 returned error can't find the container with id 70c767ebebfef417bc1f70aadd3756ce32542592c1c569a46c406bee18376b2b Jul 29 12:56:36.590702 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.590663 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:56:36.590784 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.590742 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:56:36.590827 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:36.590781 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:56:37.034034 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:37.034000 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:37.490188 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:37.490147 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" event={"ID":"92d1f840-a06b-4eb4-82ef-bd8538f83e7a","Type":"ContainerStarted","Data":"abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4"} Jul 29 12:56:37.490347 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:37.490204 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" event={"ID":"92d1f840-a06b-4eb4-82ef-bd8538f83e7a","Type":"ContainerStarted","Data":"70c767ebebfef417bc1f70aadd3756ce32542592c1c569a46c406bee18376b2b"} Jul 29 12:56:37.513300 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:37.513251 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podStartSLOduration=1.513236585 podStartE2EDuration="1.513236585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:56:37.51087455 +0000 UTC m=+1870.414071230" watchObservedRunningTime="2026-07-29 12:56:37.513236585 +0000 UTC m=+1870.416433260" Jul 29 12:56:38.451423 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:38.451390 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:38.455990 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:38.455967 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:38.494386 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:38.494354 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:38.495109 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:38.495090 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:38.860296 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:38.860258 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:38.945112 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:38.945077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:39.012702 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.012673 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:39.313122 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.313089 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:39.536049 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.536017 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:39.633206 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.633119 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:56:39.637055 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.637031 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs"] Jul 29 12:56:39.640413 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.640398 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.642973 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.642955 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-3cf3b144\"" Jul 29 12:56:39.643138 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.643114 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-3cf3b144-dockercfg-b48dk\"" Jul 29 12:56:39.658922 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.658901 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs"] Jul 29 12:56:39.667497 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.667447 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8455e5fd-0135-4d23-84e2-51a423df066c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.667597 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.667528 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wptvz\" (UniqueName: \"kubernetes.io/projected/8455e5fd-0135-4d23-84e2-51a423df066c-kube-api-access-wptvz\") pod \"payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.672220 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.672201 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28"] Jul 29 12:56:39.675583 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.675568 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.686176 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.686157 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28"] Jul 29 12:56:39.768799 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.768759 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-plugins-config-volume\") pod \"payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.768991 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.768816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8455e5fd-0135-4d23-84e2-51a423df066c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.768991 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.768845 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wptvz\" (UniqueName: \"kubernetes.io/projected/8455e5fd-0135-4d23-84e2-51a423df066c-kube-api-access-wptvz\") pod \"payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.768991 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.768937 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6hjv\" (UniqueName: \"kubernetes.io/projected/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-kube-api-access-f6hjv\") pod \"payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.769753 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.769727 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8455e5fd-0135-4d23-84e2-51a423df066c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.776386 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.776359 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wptvz\" (UniqueName: \"kubernetes.io/projected/8455e5fd-0135-4d23-84e2-51a423df066c-kube-api-access-wptvz\") pod \"payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.869803 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.869770 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-plugins-config-volume\") pod \"payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.869929 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.869858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6hjv\" (UniqueName: \"kubernetes.io/projected/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-kube-api-access-f6hjv\") pod \"payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.870709 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.870672 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-plugins-config-volume\") pod \"payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.877838 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.877812 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6hjv\" (UniqueName: \"kubernetes.io/projected/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-kube-api-access-f6hjv\") pod \"payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:39.951255 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.951197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:39.984773 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:39.984738 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:40.321645 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.321621 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs"] Jul 29 12:56:40.323604 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:56:40.323579 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8455e5fd_0135_4d23_84e2_51a423df066c.slice/crio-2c86595ef241943dc5af7292220c9e5033fb3a00fefcd7382c25cd86f4e0a146 WatchSource:0}: Error finding container 2c86595ef241943dc5af7292220c9e5033fb3a00fefcd7382c25cd86f4e0a146: Status 404 returned error can't find the container with id 2c86595ef241943dc5af7292220c9e5033fb3a00fefcd7382c25cd86f4e0a146 Jul 29 12:56:40.336426 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.336399 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28"] Jul 29 12:56:40.339537 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:56:40.339513 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3873e738_c8d2_4696_9e8a_f9d60e3ac8bf.slice/crio-66a3dac34a6d830a0a6021421cb43fe40477c3729eede72a2c9badf30a2eabb7 WatchSource:0}: Error finding container 66a3dac34a6d830a0a6021421cb43fe40477c3729eede72a2c9badf30a2eabb7: Status 404 returned error can't find the container with id 66a3dac34a6d830a0a6021421cb43fe40477c3729eede72a2c9badf30a2eabb7 Jul 29 12:56:40.506702 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.506649 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" event={"ID":"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf","Type":"ContainerStarted","Data":"c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e"} Jul 29 12:56:40.506853 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.506733 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:40.506853 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.506759 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" event={"ID":"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf","Type":"ContainerStarted","Data":"66a3dac34a6d830a0a6021421cb43fe40477c3729eede72a2c9badf30a2eabb7"} Jul 29 12:56:40.508169 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.508140 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" event={"ID":"8455e5fd-0135-4d23-84e2-51a423df066c","Type":"ContainerStarted","Data":"f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090"} Jul 29 12:56:40.508309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.508192 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:40.508309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.508214 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" event={"ID":"8455e5fd-0135-4d23-84e2-51a423df066c","Type":"ContainerStarted","Data":"2c86595ef241943dc5af7292220c9e5033fb3a00fefcd7382c25cd86f4e0a146"} Jul 29 12:56:40.526067 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.526017 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" podStartSLOduration=1.5260008539999999 podStartE2EDuration="1.526000854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:56:40.523745729 +0000 UTC m=+1873.426942407" watchObservedRunningTime="2026-07-29 12:56:40.526000854 +0000 UTC m=+1873.429197529" Jul 29 12:56:40.539002 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:40.538964 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" podStartSLOduration=1.538952366 podStartE2EDuration="1.538952366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:56:40.538387592 +0000 UTC m=+1873.441584281" watchObservedRunningTime="2026-07-29 12:56:40.538952366 +0000 UTC m=+1873.442149041" Jul 29 12:56:51.515180 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:51.515151 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:56:51.515575 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:51.515233 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:56:54.085442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.084229 2563 patch_prober.go:28] interesting pod/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": read tcp 10.134.0.2:57164->10.134.0.34:15021: read: connection reset by peer" start-of-body= Jul 29 12:56:54.085442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.084304 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": read tcp 10.134.0.2:57164->10.134.0.34:15021: read: connection reset by peer" Jul 29 12:56:54.572797 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.572759 2563 generic.go:356] "Generic (PLEG): container finished" podID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerID="abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4" exitCode=0 Jul 29 12:56:54.572995 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.572834 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" event={"ID":"92d1f840-a06b-4eb4-82ef-bd8538f83e7a","Type":"ContainerDied","Data":"abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4"} Jul 29 12:56:54.573261 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.573246 2563 scope.go:117] "RemoveContainer" containerID="abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4" Jul 29 12:56:54.574168 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.574144 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:56:54.574226 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.574207 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:56:54.574263 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:54.574236 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:56:55.578562 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:55.578524 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" event={"ID":"92d1f840-a06b-4eb4-82ef-bd8538f83e7a","Type":"ContainerStarted","Data":"22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31"} Jul 29 12:56:56.451472 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:56.451424 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:56:56.451690 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:56.451650 2563 patch_prober.go:28] interesting pod/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 29 12:56:56.451765 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:56.451718 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 29 12:56:57.451278 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:57.451247 2563 patch_prober.go:28] interesting pod/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 29 12:56:57.451743 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:57.451300 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 29 12:56:58.451307 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:58.451269 2563 patch_prober.go:28] interesting pod/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 29 12:56:58.451704 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:58.451356 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 29 12:56:59.451151 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:59.451111 2563 patch_prober.go:28] interesting pod/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 29 12:56:59.451320 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:56:59.451181 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 29 12:57:00.451789 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:00.451752 2563 patch_prober.go:28] interesting pod/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 29 12:57:00.452170 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:00.451835 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 29 12:57:01.454672 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:01.454642 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:57:01.455283 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:01.454897 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:57:01.455719 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:01.455703 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:57:09.646360 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.646320 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:09.672987 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.672951 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28"] Jul 29 12:57:09.673206 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.673168 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" podUID="3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" containerName="payload-processing" containerID="cri-o://c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e" gracePeriod=30 Jul 29 12:57:09.678705 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.678681 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs"] Jul 29 12:57:09.679039 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.678992 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" podUID="8455e5fd-0135-4d23-84e2-51a423df066c" containerName="payload-pre-processing" containerID="cri-o://f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090" gracePeriod=30 Jul 29 12:57:09.715583 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.715543 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:09.773903 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.773875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:09.828763 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.828696 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:09.934445 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.934415 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj"] Jul 29 12:57:09.934685 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.934650 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" containerID="cri-o://22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31" gracePeriod=30 Jul 29 12:57:09.940771 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.940745 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:57:09.960622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.960571 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:09.972124 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:09.972103 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:57:10.027769 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.027737 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-plugins-config-volume\") pod \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " Jul 29 12:57:10.027945 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.027825 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6hjv\" (UniqueName: \"kubernetes.io/projected/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-kube-api-access-f6hjv\") pod \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\" (UID: \"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf\") " Jul 29 12:57:10.028109 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.028078 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" (UID: "3873e738-c8d2-4696-9e8a-f9d60e3ac8bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:57:10.030008 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.029986 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-kube-api-access-f6hjv" (OuterVolumeSpecName: "kube-api-access-f6hjv") pod "3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" (UID: "3873e738-c8d2-4696-9e8a-f9d60e3ac8bf"). InnerVolumeSpecName "kube-api-access-f6hjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:10.129428 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.129395 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8455e5fd-0135-4d23-84e2-51a423df066c-plugins-config-volume\") pod \"8455e5fd-0135-4d23-84e2-51a423df066c\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " Jul 29 12:57:10.129612 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.129451 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wptvz\" (UniqueName: \"kubernetes.io/projected/8455e5fd-0135-4d23-84e2-51a423df066c-kube-api-access-wptvz\") pod \"8455e5fd-0135-4d23-84e2-51a423df066c\" (UID: \"8455e5fd-0135-4d23-84e2-51a423df066c\") " Jul 29 12:57:10.129699 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.129686 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6hjv\" (UniqueName: \"kubernetes.io/projected/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-kube-api-access-f6hjv\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:10.129735 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.129703 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:10.129848 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.129821 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8455e5fd-0135-4d23-84e2-51a423df066c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8455e5fd-0135-4d23-84e2-51a423df066c" (UID: "8455e5fd-0135-4d23-84e2-51a423df066c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:57:10.131497 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.131476 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8455e5fd-0135-4d23-84e2-51a423df066c-kube-api-access-wptvz" (OuterVolumeSpecName: "kube-api-access-wptvz") pod "8455e5fd-0135-4d23-84e2-51a423df066c" (UID: "8455e5fd-0135-4d23-84e2-51a423df066c"). InnerVolumeSpecName "kube-api-access-wptvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:10.204505 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.204415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:10.230635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.230608 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8455e5fd-0135-4d23-84e2-51a423df066c-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:10.230635 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.230630 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wptvz\" (UniqueName: \"kubernetes.io/projected/8455e5fd-0135-4d23-84e2-51a423df066c-kube-api-access-wptvz\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:10.444505 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.444441 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:10.544287 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.544243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:10.651983 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.651951 2563 generic.go:356] "Generic (PLEG): container finished" podID="8455e5fd-0135-4d23-84e2-51a423df066c" containerID="f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090" exitCode=0 Jul 29 12:57:10.652406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.651991 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" Jul 29 12:57:10.652406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.652036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" event={"ID":"8455e5fd-0135-4d23-84e2-51a423df066c","Type":"ContainerDied","Data":"f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090"} Jul 29 12:57:10.652406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.652088 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs" event={"ID":"8455e5fd-0135-4d23-84e2-51a423df066c","Type":"ContainerDied","Data":"2c86595ef241943dc5af7292220c9e5033fb3a00fefcd7382c25cd86f4e0a146"} Jul 29 12:57:10.652406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.652110 2563 scope.go:117] "RemoveContainer" containerID="f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090" Jul 29 12:57:10.653382 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.653358 2563 generic.go:356] "Generic (PLEG): container finished" podID="3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" containerID="c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e" exitCode=0 Jul 29 12:57:10.653508 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.653391 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" event={"ID":"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf","Type":"ContainerDied","Data":"c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e"} Jul 29 12:57:10.653508 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.653407 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" Jul 29 12:57:10.653508 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.653429 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28" event={"ID":"3873e738-c8d2-4696-9e8a-f9d60e3ac8bf","Type":"ContainerDied","Data":"66a3dac34a6d830a0a6021421cb43fe40477c3729eede72a2c9badf30a2eabb7"} Jul 29 12:57:10.656855 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.656809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:10.661516 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.661498 2563 scope.go:117] "RemoveContainer" containerID="f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090" Jul 29 12:57:10.661866 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:57:10.661837 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090\": container with ID starting with f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090 not found: ID does not exist" containerID="f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090" Jul 29 12:57:10.661962 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.661869 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090"} err="failed to get container status \"f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090\": rpc error: code = NotFound desc = could not find container \"f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090\": container with ID starting with f42428fa35528ed85d7cb79728f4762b1cd6fe1e3131b9b6a62fe87d6de1c090 not found: ID does not exist" Jul 29 12:57:10.661962 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.661891 2563 scope.go:117] "RemoveContainer" containerID="c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e" Jul 29 12:57:10.669816 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.669799 2563 scope.go:117] "RemoveContainer" containerID="c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e" Jul 29 12:57:10.670050 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:57:10.670032 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e\": container with ID starting with c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e not found: ID does not exist" containerID="c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e" Jul 29 12:57:10.670103 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.670058 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e"} err="failed to get container status \"c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e\": rpc error: code = NotFound desc = could not find container \"c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e\": container with ID starting with c2172f6afa05d0167371a77e5e2079c49eebc1a1893901fc7dcacbea610c893e not found: ID does not exist" Jul 29 12:57:10.681797 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.681772 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28"] Jul 29 12:57:10.686008 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.685971 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-3cf3b144-75f9b85bd8-gzt28"] Jul 29 12:57:10.699292 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.699269 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs"] Jul 29 12:57:10.702897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.702875 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-3cf3b144-66bc8fc4b4-qs7xs"] Jul 29 12:57:10.946960 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.946878 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw"] Jul 29 12:57:10.947428 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.947405 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" containerName="payload-processing" Jul 29 12:57:10.947428 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.947427 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" containerName="payload-processing" Jul 29 12:57:10.947643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.947448 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8455e5fd-0135-4d23-84e2-51a423df066c" containerName="payload-pre-processing" Jul 29 12:57:10.947643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.947472 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8455e5fd-0135-4d23-84e2-51a423df066c" containerName="payload-pre-processing" Jul 29 12:57:10.947643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.947548 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" containerName="payload-processing" Jul 29 12:57:10.947643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.947563 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8455e5fd-0135-4d23-84e2-51a423df066c" containerName="payload-pre-processing" Jul 29 12:57:10.950755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.950734 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:10.956005 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.955984 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-95d13da9-gw-openshift-default-dockercfg-7j42b\"" Jul 29 12:57:10.963236 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.963215 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw"] Jul 29 12:57:10.984281 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:10.984234 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:11.037313 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037282 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-data\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037484 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kzts\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-kube-api-access-8kzts\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037484 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-envoy\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037484 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037413 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-certs\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037484 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/343ada02-78fd-4133-8cc7-556a4715e5fc-istiod-ca-cert\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037759 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037495 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-socket\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037759 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-podinfo\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037759 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-token\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.037759 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.037564 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-credential-socket\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138197 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-data\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138197 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138202 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kzts\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-kube-api-access-8kzts\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-envoy\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138271 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-certs\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138293 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/343ada02-78fd-4133-8cc7-556a4715e5fc-istiod-ca-cert\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-socket\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-podinfo\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138384 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-token\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138436 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-credential-socket\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-data\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138704 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-certs\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.138822 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138778 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-credential-socket\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.139181 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.138966 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-socket\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.139181 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.139074 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/343ada02-78fd-4133-8cc7-556a4715e5fc-istiod-ca-cert\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.140791 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.140768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-podinfo\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.140880 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.140827 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-envoy\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.145835 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.145814 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-token\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.146035 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.146020 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kzts\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-kube-api-access-8kzts\") pod \"e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.262598 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.262570 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:11.298275 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.298246 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:11.399796 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.399772 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw"] Jul 29 12:57:11.401581 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:57:11.401552 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod343ada02_78fd_4133_8cc7_556a4715e5fc.slice/crio-9249c2f1d186409635fbe01eddac8c2256512e19e11ca612a01805aa241f4858 WatchSource:0}: Error finding container 9249c2f1d186409635fbe01eddac8c2256512e19e11ca612a01805aa241f4858: Status 404 returned error can't find the container with id 9249c2f1d186409635fbe01eddac8c2256512e19e11ca612a01805aa241f4858 Jul 29 12:57:11.403552 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.403524 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:11.403614 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.403581 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:11.403646 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.403610 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:11.658827 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.658751 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3873e738-c8d2-4696-9e8a-f9d60e3ac8bf" path="/var/lib/kubelet/pods/3873e738-c8d2-4696-9e8a-f9d60e3ac8bf/volumes" Jul 29 12:57:11.659228 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.659206 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8455e5fd-0135-4d23-84e2-51a423df066c" path="/var/lib/kubelet/pods/8455e5fd-0135-4d23-84e2-51a423df066c/volumes" Jul 29 12:57:11.660323 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.660294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" event={"ID":"343ada02-78fd-4133-8cc7-556a4715e5fc","Type":"ContainerStarted","Data":"400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb"} Jul 29 12:57:11.660484 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.660351 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" event={"ID":"343ada02-78fd-4133-8cc7-556a4715e5fc","Type":"ContainerStarted","Data":"9249c2f1d186409635fbe01eddac8c2256512e19e11ca612a01805aa241f4858"} Jul 29 12:57:11.679683 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:11.679638 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podStartSLOduration=1.679622765 podStartE2EDuration="1.679622765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:11.678532428 +0000 UTC m=+1904.581729127" watchObservedRunningTime="2026-07-29 12:57:11.679622765 +0000 UTC m=+1904.582819440" Jul 29 12:57:12.263059 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:12.263026 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:12.267964 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:12.267935 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:12.666417 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:12.666338 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:12.667201 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:12.667182 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:13.381507 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.381449 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:13.436121 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.436084 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:13.852321 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.852289 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:13.950315 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.950279 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:13.958105 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.958083 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw"] Jul 29 12:57:13.962647 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.962625 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:13.965375 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.965355 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-95d13da9\"" Jul 29 12:57:13.972414 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:13.972395 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw"] Jul 29 12:57:14.066130 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.066104 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sr8q\" (UniqueName: \"kubernetes.io/projected/dd668a41-c251-47b0-aa42-15181897524a-kube-api-access-7sr8q\") pod \"payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.066272 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.066144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd668a41-c251-47b0-aa42-15181897524a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.093045 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.093007 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x"] Jul 29 12:57:14.097186 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.097160 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.099658 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.099632 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-95d13da9-dockercfg-rvfqf\"" Jul 29 12:57:14.106109 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.106046 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x"] Jul 29 12:57:14.167329 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.167299 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sr8q\" (UniqueName: \"kubernetes.io/projected/dd668a41-c251-47b0-aa42-15181897524a-kube-api-access-7sr8q\") pod \"payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.167521 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.167347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd668a41-c251-47b0-aa42-15181897524a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.168073 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.168055 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd668a41-c251-47b0-aa42-15181897524a-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.175276 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.175248 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sr8q\" (UniqueName: \"kubernetes.io/projected/dd668a41-c251-47b0-aa42-15181897524a-kube-api-access-7sr8q\") pod \"payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.268075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.268049 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgjfr\" (UniqueName: \"kubernetes.io/projected/7c230510-83a2-46be-88e0-f4ebdae728a1-kube-api-access-cgjfr\") pod \"payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.268198 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.268122 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c230510-83a2-46be-88e0-f4ebdae728a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.274995 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.274948 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:14.369205 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.369126 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgjfr\" (UniqueName: \"kubernetes.io/projected/7c230510-83a2-46be-88e0-f4ebdae728a1-kube-api-access-cgjfr\") pod \"payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.369348 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.369233 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c230510-83a2-46be-88e0-f4ebdae728a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.370052 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.370026 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c230510-83a2-46be-88e0-f4ebdae728a1-plugins-config-volume\") pod \"payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.379068 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.379034 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgjfr\" (UniqueName: \"kubernetes.io/projected/7c230510-83a2-46be-88e0-f4ebdae728a1-kube-api-access-cgjfr\") pod \"payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.412855 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.412831 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.540784 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.540719 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x"] Jul 29 12:57:14.542977 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:57:14.542943 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c230510_83a2_46be_88e0_f4ebdae728a1.slice/crio-7b0f25c5f385d449fd1d109191dd2dfd831b06ef42fb4fda99815dc497ab9e0c WatchSource:0}: Error finding container 7b0f25c5f385d449fd1d109191dd2dfd831b06ef42fb4fda99815dc497ab9e0c: Status 404 returned error can't find the container with id 7b0f25c5f385d449fd1d109191dd2dfd831b06ef42fb4fda99815dc497ab9e0c Jul 29 12:57:14.611325 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.611267 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw"] Jul 29 12:57:14.613600 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:57:14.613571 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd668a41_c251_47b0_aa42_15181897524a.slice/crio-21211afd039cfe16feeacd38369a3a6db6a5fc8bf27c868d3b74753d39dd51f3 WatchSource:0}: Error finding container 21211afd039cfe16feeacd38369a3a6db6a5fc8bf27c868d3b74753d39dd51f3: Status 404 returned error can't find the container with id 21211afd039cfe16feeacd38369a3a6db6a5fc8bf27c868d3b74753d39dd51f3 Jul 29 12:57:14.680269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.680243 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" event={"ID":"dd668a41-c251-47b0-aa42-15181897524a","Type":"ContainerStarted","Data":"21211afd039cfe16feeacd38369a3a6db6a5fc8bf27c868d3b74753d39dd51f3"} Jul 29 12:57:14.681875 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.681849 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" event={"ID":"7c230510-83a2-46be-88e0-f4ebdae728a1","Type":"ContainerStarted","Data":"fd74a203d98edcdd2adce563b9151e529ce070ecfdaa3dd99b5d0219e10a5711"} Jul 29 12:57:14.681969 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.681899 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" event={"ID":"7c230510-83a2-46be-88e0-f4ebdae728a1","Type":"ContainerStarted","Data":"7b0f25c5f385d449fd1d109191dd2dfd831b06ef42fb4fda99815dc497ab9e0c"} Jul 29 12:57:14.682057 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.682037 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:14.699440 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:14.699397 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" podStartSLOduration=0.699380252 podStartE2EDuration="699.380252ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:14.698210457 +0000 UTC m=+1907.601407133" watchObservedRunningTime="2026-07-29 12:57:14.699380252 +0000 UTC m=+1907.602576928" Jul 29 12:57:15.172044 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.172017 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:57:15.279653 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279625 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-credential-socket\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.279653 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279656 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-certs\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.279867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279700 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-socket\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.279867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279724 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-podinfo\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.279867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279761 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-data\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.279867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279787 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-envoy\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.279867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279828 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istiod-ca-cert\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.280115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279893 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq2bt\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-kube-api-access-lq2bt\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.280115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.279934 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-token\") pod \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\" (UID: \"92d1f840-a06b-4eb4-82ef-bd8538f83e7a\") " Jul 29 12:57:15.280115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280013 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:15.280115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280027 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:15.280299 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280263 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.280299 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280283 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.280403 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280310 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:57:15.280690 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280602 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:15.280812 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.280738 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:15.282371 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.282350 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 12:57:15.282452 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.282355 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:15.282723 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.282700 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-kube-api-access-lq2bt" (OuterVolumeSpecName: "kube-api-access-lq2bt") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "kube-api-access-lq2bt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:15.282781 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.282731 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "92d1f840-a06b-4eb4-82ef-bd8538f83e7a" (UID: "92d1f840-a06b-4eb4-82ef-bd8538f83e7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:15.381179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381147 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.381179 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381173 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.381341 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381185 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.381341 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381196 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.381341 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381209 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq2bt\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-kube-api-access-lq2bt\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.381341 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381222 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.381341 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.381234 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92d1f840-a06b-4eb4-82ef-bd8538f83e7a-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:15.688776 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.688744 2563 generic.go:356] "Generic (PLEG): container finished" podID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerID="22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31" exitCode=0 Jul 29 12:57:15.688919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.688801 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" Jul 29 12:57:15.688919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.688801 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" event={"ID":"92d1f840-a06b-4eb4-82ef-bd8538f83e7a","Type":"ContainerDied","Data":"22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31"} Jul 29 12:57:15.688919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.688857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj" event={"ID":"92d1f840-a06b-4eb4-82ef-bd8538f83e7a","Type":"ContainerDied","Data":"70c767ebebfef417bc1f70aadd3756ce32542592c1c569a46c406bee18376b2b"} Jul 29 12:57:15.688919 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.688882 2563 scope.go:117] "RemoveContainer" containerID="22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31" Jul 29 12:57:15.691278 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.690697 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" event={"ID":"dd668a41-c251-47b0-aa42-15181897524a","Type":"ContainerStarted","Data":"57b5982e8119c95f2343d6f77f94b1fcf45c5dca80b3138bfb8888ba51694043"} Jul 29 12:57:15.691278 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.690968 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:15.698870 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.698853 2563 scope.go:117] "RemoveContainer" containerID="abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4" Jul 29 12:57:15.706291 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.706271 2563 scope.go:117] "RemoveContainer" containerID="22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31" Jul 29 12:57:15.706555 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:57:15.706539 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31\": container with ID starting with 22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31 not found: ID does not exist" containerID="22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31" Jul 29 12:57:15.706626 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.706562 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31"} err="failed to get container status \"22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31\": rpc error: code = NotFound desc = could not find container \"22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31\": container with ID starting with 22c7701855886ec6831d8d25fa122e901d33119b0b55c61737a2c5928eb20c31 not found: ID does not exist" Jul 29 12:57:15.706626 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.706578 2563 scope.go:117] "RemoveContainer" containerID="abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4" Jul 29 12:57:15.706793 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:57:15.706779 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4\": container with ID starting with abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4 not found: ID does not exist" containerID="abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4" Jul 29 12:57:15.706830 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.706797 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4"} err="failed to get container status \"abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4\": rpc error: code = NotFound desc = could not find container \"abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4\": container with ID starting with abc1d282911581bd65519bb825fa9c58984d23a5a4ae6c38f75ff64365fe7df4 not found: ID does not exist" Jul 29 12:57:15.710886 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.710840 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" podStartSLOduration=2.710829235 podStartE2EDuration="2.710829235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:15.709100664 +0000 UTC m=+1908.612297538" watchObservedRunningTime="2026-07-29 12:57:15.710829235 +0000 UTC m=+1908.614025910" Jul 29 12:57:15.722625 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.722597 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj"] Jul 29 12:57:15.726172 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:15.726152 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3cf3b144-openshift-default-7fb85c7d48-w4sdj"] Jul 29 12:57:17.658302 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:17.658259 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" path="/var/lib/kubelet/pods/92d1f840-a06b-4eb4-82ef-bd8538f83e7a/volumes" Jul 29 12:57:25.693610 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:25.693576 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:26.699067 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:26.699033 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:27.667979 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.667902 2563 patch_prober.go:28] interesting pod/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 29 12:57:27.667979 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.667953 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 29 12:57:27.746042 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.746011 2563 generic.go:356] "Generic (PLEG): container finished" podID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerID="400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb" exitCode=0 Jul 29 12:57:27.746339 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.746084 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" event={"ID":"343ada02-78fd-4133-8cc7-556a4715e5fc","Type":"ContainerDied","Data":"400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb"} Jul 29 12:57:27.746473 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.746444 2563 scope.go:117] "RemoveContainer" containerID="400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb" Jul 29 12:57:27.747328 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.747303 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:27.747396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.747364 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:27.747396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:27.747392 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:28.752941 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:28.752910 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" event={"ID":"343ada02-78fd-4133-8cc7-556a4715e5fc","Type":"ContainerStarted","Data":"e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7"} Jul 29 12:57:29.263869 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:29.263828 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:29.264200 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:29.264176 2563 patch_prober.go:28] interesting pod/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 29 12:57:29.264387 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:29.264361 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 29 12:57:30.263266 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:30.263233 2563 patch_prober.go:28] interesting pod/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 29 12:57:30.263859 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:30.263812 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 29 12:57:31.263093 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:31.263052 2563 patch_prober.go:28] interesting pod/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 29 12:57:31.263289 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:31.263160 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 29 12:57:32.263755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:32.263710 2563 patch_prober.go:28] interesting pod/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 29 12:57:32.264238 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:32.263784 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 29 12:57:33.267213 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:33.267180 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:33.267659 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:33.267496 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:33.268232 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:33.268211 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:44.565561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.565520 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:44.610687 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.610654 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x"] Jul 29 12:57:44.610929 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.610903 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" podUID="7c230510-83a2-46be-88e0-f4ebdae728a1" containerName="payload-processing" containerID="cri-o://fd74a203d98edcdd2adce563b9151e529ce070ecfdaa3dd99b5d0219e10a5711" gracePeriod=30 Jul 29 12:57:44.616699 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.616673 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw"] Jul 29 12:57:44.616924 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.616883 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" podUID="dd668a41-c251-47b0-aa42-15181897524a" containerName="payload-pre-processing" containerID="cri-o://57b5982e8119c95f2343d6f77f94b1fcf45c5dca80b3138bfb8888ba51694043" gracePeriod=30 Jul 29 12:57:44.656433 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.656396 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:44.724562 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.724535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:44.794552 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.794411 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:44.831811 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.831780 2563 generic.go:356] "Generic (PLEG): container finished" podID="7c230510-83a2-46be-88e0-f4ebdae728a1" containerID="fd74a203d98edcdd2adce563b9151e529ce070ecfdaa3dd99b5d0219e10a5711" exitCode=0 Jul 29 12:57:44.831944 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.831828 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" event={"ID":"7c230510-83a2-46be-88e0-f4ebdae728a1","Type":"ContainerDied","Data":"fd74a203d98edcdd2adce563b9151e529ce070ecfdaa3dd99b5d0219e10a5711"} Jul 29 12:57:44.833620 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.833594 2563 generic.go:356] "Generic (PLEG): container finished" podID="dd668a41-c251-47b0-aa42-15181897524a" containerID="57b5982e8119c95f2343d6f77f94b1fcf45c5dca80b3138bfb8888ba51694043" exitCode=0 Jul 29 12:57:44.833713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.833630 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" event={"ID":"dd668a41-c251-47b0-aa42-15181897524a","Type":"ContainerDied","Data":"57b5982e8119c95f2343d6f77f94b1fcf45c5dca80b3138bfb8888ba51694043"} Jul 29 12:57:44.888480 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.887945 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw"] Jul 29 12:57:44.888480 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.888195 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" containerID="cri-o://e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7" gracePeriod=30 Jul 29 12:57:44.906364 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.906261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:44.931917 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.931896 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:44.934971 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:44.934952 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:45.045753 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.045722 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd668a41-c251-47b0-aa42-15181897524a-plugins-config-volume\") pod \"dd668a41-c251-47b0-aa42-15181897524a\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " Jul 29 12:57:45.045753 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.045763 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgjfr\" (UniqueName: \"kubernetes.io/projected/7c230510-83a2-46be-88e0-f4ebdae728a1-kube-api-access-cgjfr\") pod \"7c230510-83a2-46be-88e0-f4ebdae728a1\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " Jul 29 12:57:45.046019 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.045802 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sr8q\" (UniqueName: \"kubernetes.io/projected/dd668a41-c251-47b0-aa42-15181897524a-kube-api-access-7sr8q\") pod \"dd668a41-c251-47b0-aa42-15181897524a\" (UID: \"dd668a41-c251-47b0-aa42-15181897524a\") " Jul 29 12:57:45.046019 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.045840 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c230510-83a2-46be-88e0-f4ebdae728a1-plugins-config-volume\") pod \"7c230510-83a2-46be-88e0-f4ebdae728a1\" (UID: \"7c230510-83a2-46be-88e0-f4ebdae728a1\") " Jul 29 12:57:45.046175 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.046147 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd668a41-c251-47b0-aa42-15181897524a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd668a41-c251-47b0-aa42-15181897524a" (UID: "dd668a41-c251-47b0-aa42-15181897524a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:57:45.046245 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.046220 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c230510-83a2-46be-88e0-f4ebdae728a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c230510-83a2-46be-88e0-f4ebdae728a1" (UID: "7c230510-83a2-46be-88e0-f4ebdae728a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:57:45.047877 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.047855 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c230510-83a2-46be-88e0-f4ebdae728a1-kube-api-access-cgjfr" (OuterVolumeSpecName: "kube-api-access-cgjfr") pod "7c230510-83a2-46be-88e0-f4ebdae728a1" (UID: "7c230510-83a2-46be-88e0-f4ebdae728a1"). InnerVolumeSpecName "kube-api-access-cgjfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:45.047978 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.047957 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd668a41-c251-47b0-aa42-15181897524a-kube-api-access-7sr8q" (OuterVolumeSpecName: "kube-api-access-7sr8q") pod "dd668a41-c251-47b0-aa42-15181897524a" (UID: "dd668a41-c251-47b0-aa42-15181897524a"). InnerVolumeSpecName "kube-api-access-7sr8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:45.139232 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.139143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:45.147307 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.147283 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd668a41-c251-47b0-aa42-15181897524a-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:45.147307 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.147307 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgjfr\" (UniqueName: \"kubernetes.io/projected/7c230510-83a2-46be-88e0-f4ebdae728a1-kube-api-access-cgjfr\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:45.147485 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.147319 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sr8q\" (UniqueName: \"kubernetes.io/projected/dd668a41-c251-47b0-aa42-15181897524a-kube-api-access-7sr8q\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:45.147485 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.147328 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c230510-83a2-46be-88e0-f4ebdae728a1-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:45.372805 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.372765 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:45.734728 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.734692 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq"] Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735030 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735041 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735048 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c230510-83a2-46be-88e0-f4ebdae728a1" containerName="payload-processing" Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735053 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c230510-83a2-46be-88e0-f4ebdae728a1" containerName="payload-processing" Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735063 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735069 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" Jul 29 12:57:45.735071 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735075 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd668a41-c251-47b0-aa42-15181897524a" containerName="payload-pre-processing" Jul 29 12:57:45.735309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735080 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd668a41-c251-47b0-aa42-15181897524a" containerName="payload-pre-processing" Jul 29 12:57:45.735309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735145 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" Jul 29 12:57:45.735309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735152 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d1f840-a06b-4eb4-82ef-bd8538f83e7a" containerName="istio-proxy" Jul 29 12:57:45.735309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735161 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c230510-83a2-46be-88e0-f4ebdae728a1" containerName="payload-processing" Jul 29 12:57:45.735309 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.735168 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd668a41-c251-47b0-aa42-15181897524a" containerName="payload-pre-processing" Jul 29 12:57:45.738206 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.738191 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.741679 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.741651 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-329c04f9-openshift-default-dockercfg-85fjc\"" Jul 29 12:57:45.757151 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.757112 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq"] Jul 29 12:57:45.838834 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.838807 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" Jul 29 12:57:45.838834 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.838821 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x" event={"ID":"7c230510-83a2-46be-88e0-f4ebdae728a1","Type":"ContainerDied","Data":"7b0f25c5f385d449fd1d109191dd2dfd831b06ef42fb4fda99815dc497ab9e0c"} Jul 29 12:57:45.839089 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.838884 2563 scope.go:117] "RemoveContainer" containerID="fd74a203d98edcdd2adce563b9151e529ce070ecfdaa3dd99b5d0219e10a5711" Jul 29 12:57:45.840302 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.840162 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" event={"ID":"dd668a41-c251-47b0-aa42-15181897524a","Type":"ContainerDied","Data":"21211afd039cfe16feeacd38369a3a6db6a5fc8bf27c868d3b74753d39dd51f3"} Jul 29 12:57:45.840302 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.840206 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw" Jul 29 12:57:45.847298 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.847281 2563 scope.go:117] "RemoveContainer" containerID="57b5982e8119c95f2343d6f77f94b1fcf45c5dca80b3138bfb8888ba51694043" Jul 29 12:57:45.853923 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.853898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-podinfo\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854017 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.853934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx54k\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-kube-api-access-vx54k\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854017 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.853968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-envoy\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854017 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.853992 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2eca81-a710-4509-9cb6-b4d86720743c-istiod-ca-cert\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854180 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.854033 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-certs\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854180 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.854113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-token\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854180 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.854152 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-socket\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854180 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.854176 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-data\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.854351 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.854206 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-credential-socket\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.858897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.858874 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x"] Jul 29 12:57:45.862152 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.862129 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-95d13da9-7df645c6b5-22z5x"] Jul 29 12:57:45.873353 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.873328 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw"] Jul 29 12:57:45.874779 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.874760 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-95d13da9-58d87984b4-l4rhw"] Jul 29 12:57:45.955430 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955339 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-certs\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955430 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955378 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-token\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955430 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-socket\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955430 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955415 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-data\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955440 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-credential-socket\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-podinfo\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx54k\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-kube-api-access-vx54k\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955577 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-envoy\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.955778 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.955606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2eca81-a710-4509-9cb6-b4d86720743c-istiod-ca-cert\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.958274 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.958209 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-socket\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.958274 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.958246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-data\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.958439 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.958338 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-credential-socket\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.958588 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.958564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-podinfo\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.958666 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.958650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2eca81-a710-4509-9cb6-b4d86720743c-istiod-ca-cert\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.958730 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.958710 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-certs\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.960032 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.960016 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-envoy\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.963562 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.963540 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-token\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:45.963711 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:45.963691 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx54k\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-kube-api-access-vx54k\") pod \"e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:46.048075 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.048039 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:46.185040 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.185014 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq"] Jul 29 12:57:46.187172 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:57:46.187124 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2eca81_a710_4509_9cb6_b4d86720743c.slice/crio-5c24cb901d54776deb01ad15b7ddcae53c4cf6e38253585894fa7d6e9efe6bfa WatchSource:0}: Error finding container 5c24cb901d54776deb01ad15b7ddcae53c4cf6e38253585894fa7d6e9efe6bfa: Status 404 returned error can't find the container with id 5c24cb901d54776deb01ad15b7ddcae53c4cf6e38253585894fa7d6e9efe6bfa Jul 29 12:57:46.187290 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.187268 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:46.189226 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.189194 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:46.189294 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.189261 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:46.189343 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.189299 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:57:46.287641 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.287598 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:46.846864 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.846831 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" event={"ID":"2e2eca81-a710-4509-9cb6-b4d86720743c","Type":"ContainerStarted","Data":"92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1"} Jul 29 12:57:46.847257 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.846887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" event={"ID":"2e2eca81-a710-4509-9cb6-b4d86720743c","Type":"ContainerStarted","Data":"5c24cb901d54776deb01ad15b7ddcae53c4cf6e38253585894fa7d6e9efe6bfa"} Jul 29 12:57:46.867360 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:46.867309 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podStartSLOduration=1.86729185 podStartE2EDuration="1.86729185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:46.86591531 +0000 UTC m=+1939.769111986" watchObservedRunningTime="2026-07-29 12:57:46.86729185 +0000 UTC m=+1939.770488526" Jul 29 12:57:47.048396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:47.048357 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:47.052896 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:47.052870 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:47.659062 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:47.659032 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c230510-83a2-46be-88e0-f4ebdae728a1" path="/var/lib/kubelet/pods/7c230510-83a2-46be-88e0-f4ebdae728a1/volumes" Jul 29 12:57:47.659390 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:47.659375 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd668a41-c251-47b0-aa42-15181897524a" path="/var/lib/kubelet/pods/dd668a41-c251-47b0-aa42-15181897524a/volumes" Jul 29 12:57:47.852559 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:47.852531 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:47.853690 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:47.853670 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:57:48.218988 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.218951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:48.270288 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.270248 2563 patch_prober.go:28] interesting pod/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 12:57:48.270442 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.270318 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 12:57:48.324766 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.324730 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:48.419651 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.419617 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:48.612644 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.612549 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:48.903323 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.903235 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:48.979775 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.979737 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l"] Jul 29 12:57:48.982045 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.982009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:57:48.984326 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.984308 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:48.986674 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.986657 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-329c04f9\"" Jul 29 12:57:48.994026 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:48.994007 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l"] Jul 29 12:57:49.019621 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.019584 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z"] Jul 29 12:57:49.023023 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.022999 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.025570 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.025543 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-329c04f9-dockercfg-vv4dt\"" Jul 29 12:57:49.039502 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.039468 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z"] Jul 29 12:57:49.085588 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.085553 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9flzp\" (UniqueName: \"kubernetes.io/projected/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-kube-api-access-9flzp\") pod \"payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.085762 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.085616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.187193 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.187086 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9flzp\" (UniqueName: \"kubernetes.io/projected/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-kube-api-access-9flzp\") pod \"payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.187193 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.187174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-plugins-config-volume\") pod \"payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.187377 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.187218 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kswkf\" (UniqueName: \"kubernetes.io/projected/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-kube-api-access-kswkf\") pod \"payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.187377 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.187246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.187916 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.187898 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.195607 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.195574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9flzp\" (UniqueName: \"kubernetes.io/projected/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-kube-api-access-9flzp\") pod \"payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.287866 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.287827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-plugins-config-volume\") pod \"payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.288060 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.287882 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kswkf\" (UniqueName: \"kubernetes.io/projected/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-kube-api-access-kswkf\") pod \"payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.288680 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.288660 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-plugins-config-volume\") pod \"payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.296199 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.296169 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.297146 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.297121 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kswkf\" (UniqueName: \"kubernetes.io/projected/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-kube-api-access-kswkf\") pod \"payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.333109 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.333071 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:49.483847 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.483818 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z"] Jul 29 12:57:49.485760 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:57:49.485728 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0b33fb_1a08_48b3_b239_e1175bd36ff8.slice/crio-960091ef2850f20b34fd47ec1f199ee9f1fec3429652cd423e7f30f4702ae39f WatchSource:0}: Error finding container 960091ef2850f20b34fd47ec1f199ee9f1fec3429652cd423e7f30f4702ae39f: Status 404 returned error can't find the container with id 960091ef2850f20b34fd47ec1f199ee9f1fec3429652cd423e7f30f4702ae39f Jul 29 12:57:49.657976 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:57:49.657947 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b5657a_a2f5_4e39_9899_7d2eee94c2e0.slice/crio-d3ae7847438da282165aeea3b02ea94a91961a178c05d7a5d0f30fd3578065a4 WatchSource:0}: Error finding container d3ae7847438da282165aeea3b02ea94a91961a178c05d7a5d0f30fd3578065a4: Status 404 returned error can't find the container with id d3ae7847438da282165aeea3b02ea94a91961a178c05d7a5d0f30fd3578065a4 Jul 29 12:57:49.663222 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.663194 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l"] Jul 29 12:57:49.866540 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.866427 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" event={"ID":"13b5657a-a2f5-4e39-9899-7d2eee94c2e0","Type":"ContainerStarted","Data":"7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb"} Jul 29 12:57:49.866540 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.866510 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" event={"ID":"13b5657a-a2f5-4e39-9899-7d2eee94c2e0","Type":"ContainerStarted","Data":"d3ae7847438da282165aeea3b02ea94a91961a178c05d7a5d0f30fd3578065a4"} Jul 29 12:57:49.866750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.866603 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:57:49.868155 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.868131 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" event={"ID":"3d0b33fb-1a08-48b3-b239-e1175bd36ff8","Type":"ContainerStarted","Data":"3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5"} Jul 29 12:57:49.868281 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.868173 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" event={"ID":"3d0b33fb-1a08-48b3-b239-e1175bd36ff8","Type":"ContainerStarted","Data":"960091ef2850f20b34fd47ec1f199ee9f1fec3429652cd423e7f30f4702ae39f"} Jul 29 12:57:49.885847 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.885790 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" podStartSLOduration=1.885773052 podStartE2EDuration="1.885773052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:49.884030926 +0000 UTC m=+1942.787227604" watchObservedRunningTime="2026-07-29 12:57:49.885773052 +0000 UTC m=+1942.788969726" Jul 29 12:57:49.902374 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:49.902320 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" podStartSLOduration=0.902302537 podStartE2EDuration="902.302537ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:57:49.900300218 +0000 UTC m=+1942.803496895" watchObservedRunningTime="2026-07-29 12:57:49.902302537 +0000 UTC m=+1942.805499212" Jul 29 12:57:50.139082 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.139058 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:50.297891 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.297854 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-data\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298091 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.297907 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/343ada02-78fd-4133-8cc7-556a4715e5fc-istiod-ca-cert\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298091 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.297939 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-certs\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298091 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.297961 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-token\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298091 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298037 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-socket\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298091 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298069 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-credential-socket\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298361 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298113 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kzts\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-kube-api-access-8kzts\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298361 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298136 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-envoy\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298361 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298212 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-podinfo\") pod \"343ada02-78fd-4133-8cc7-556a4715e5fc\" (UID: \"343ada02-78fd-4133-8cc7-556a4715e5fc\") " Jul 29 12:57:50.298527 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298354 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/343ada02-78fd-4133-8cc7-556a4715e5fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:57:50.298575 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298538 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:50.298631 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298597 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:50.298631 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298602 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/343ada02-78fd-4133-8cc7-556a4715e5fc-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.298713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298611 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:50.298713 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.298698 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:50.300644 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.300620 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-kube-api-access-8kzts" (OuterVolumeSpecName: "kube-api-access-8kzts") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "kube-api-access-8kzts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:50.300790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.300611 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:57:50.300790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.300687 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:57:50.300901 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.300873 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "343ada02-78fd-4133-8cc7-556a4715e5fc" (UID: "343ada02-78fd-4133-8cc7-556a4715e5fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399368 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399402 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399415 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kzts\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-kube-api-access-8kzts\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399426 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399441 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399451 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399481 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399477 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/343ada02-78fd-4133-8cc7-556a4715e5fc-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.399812 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.399489 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/343ada02-78fd-4133-8cc7-556a4715e5fc-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:57:50.876392 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.876361 2563 generic.go:356] "Generic (PLEG): container finished" podID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerID="e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7" exitCode=0 Jul 29 12:57:50.876570 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.876408 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" Jul 29 12:57:50.876570 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.876446 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" event={"ID":"343ada02-78fd-4133-8cc7-556a4715e5fc","Type":"ContainerDied","Data":"e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7"} Jul 29 12:57:50.876570 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.876523 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw" event={"ID":"343ada02-78fd-4133-8cc7-556a4715e5fc","Type":"ContainerDied","Data":"9249c2f1d186409635fbe01eddac8c2256512e19e11ca612a01805aa241f4858"} Jul 29 12:57:50.876570 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.876557 2563 scope.go:117] "RemoveContainer" containerID="e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7" Jul 29 12:57:50.876992 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.876974 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:57:50.885478 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.885447 2563 scope.go:117] "RemoveContainer" containerID="400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb" Jul 29 12:57:50.892848 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.892831 2563 scope.go:117] "RemoveContainer" containerID="e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7" Jul 29 12:57:50.893090 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:57:50.893070 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7\": container with ID starting with e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7 not found: ID does not exist" containerID="e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7" Jul 29 12:57:50.893157 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.893099 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7"} err="failed to get container status \"e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7\": rpc error: code = NotFound desc = could not find container \"e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7\": container with ID starting with e9e53b836a87f87fab1223a21b983924c1d6e69ec4625bdb46b07b87159c02b7 not found: ID does not exist" Jul 29 12:57:50.893157 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.893117 2563 scope.go:117] "RemoveContainer" containerID="400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb" Jul 29 12:57:50.893334 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:57:50.893318 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb\": container with ID starting with 400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb not found: ID does not exist" containerID="400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb" Jul 29 12:57:50.893370 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.893339 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb"} err="failed to get container status \"400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb\": rpc error: code = NotFound desc = could not find container \"400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb\": container with ID starting with 400d89e6ae5c163e87616c94f7d08063161a0031bb4f20ccdd8068298ecc00cb not found: ID does not exist" Jul 29 12:57:50.898127 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.898105 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw"] Jul 29 12:57:50.902435 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:50.902415 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-95d13da9-gw-openshift-default-57fdb8c656-n9zjw"] Jul 29 12:57:51.659687 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:57:51.659657 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" path="/var/lib/kubelet/pods/343ada02-78fd-4133-8cc7-556a4715e5fc/volumes" Jul 29 12:58:00.878454 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:00.878421 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:58:01.884817 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:01.884790 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:58:03.058397 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.057967 2563 patch_prober.go:28] interesting pod/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:41964->10.134.0.40:15021: read: connection reset by peer" start-of-body= Jul 29 12:58:03.058397 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.058046 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": read tcp 10.134.0.2:41964->10.134.0.40:15021: read: connection reset by peer" Jul 29 12:58:03.936969 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.936936 2563 generic.go:356] "Generic (PLEG): container finished" podID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerID="92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1" exitCode=0 Jul 29 12:58:03.937125 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.936987 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" event={"ID":"2e2eca81-a710-4509-9cb6-b4d86720743c","Type":"ContainerDied","Data":"92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1"} Jul 29 12:58:03.937389 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.937372 2563 scope.go:117] "RemoveContainer" containerID="92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1" Jul 29 12:58:03.938408 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.938383 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:03.938475 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.938449 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:03.938518 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:03.938494 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:04.944024 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:04.943990 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" event={"ID":"2e2eca81-a710-4509-9cb6-b4d86720743c","Type":"ContainerStarted","Data":"95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1"} Jul 29 12:58:05.048396 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:05.048360 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:58:05.048712 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:05.048687 2563 patch_prober.go:28] interesting pod/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 29 12:58:05.048800 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:05.048745 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 29 12:58:06.049204 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:06.049167 2563 patch_prober.go:28] interesting pod/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 29 12:58:06.049593 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:06.049224 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 29 12:58:07.049037 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:07.049006 2563 patch_prober.go:28] interesting pod/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 29 12:58:07.049216 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:07.049058 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 29 12:58:08.049000 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:08.048965 2563 patch_prober.go:28] interesting pod/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 29 12:58:08.049372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:08.049026 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 29 12:58:09.052334 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:09.052304 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:58:09.052782 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:09.052571 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:58:09.053281 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:09.053265 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:58:19.146815 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.146779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:19.175565 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.175532 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z"] Jul 29 12:58:19.175836 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.175810 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" podUID="3d0b33fb-1a08-48b3-b239-e1175bd36ff8" containerName="payload-processing" containerID="cri-o://3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5" gracePeriod=30 Jul 29 12:58:19.184043 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.183894 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l"] Jul 29 12:58:19.184175 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.184096 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" podUID="13b5657a-a2f5-4e39-9899-7d2eee94c2e0" containerName="payload-pre-processing" containerID="cri-o://7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb" gracePeriod=30 Jul 29 12:58:19.221395 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.221364 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:19.296927 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.296884 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:19.433051 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.433024 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:58:19.462248 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.462219 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:58:19.559863 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.559827 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9flzp\" (UniqueName: \"kubernetes.io/projected/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-kube-api-access-9flzp\") pod \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " Jul 29 12:58:19.559863 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.559866 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-plugins-config-volume\") pod \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\" (UID: \"13b5657a-a2f5-4e39-9899-7d2eee94c2e0\") " Jul 29 12:58:19.560114 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.559911 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kswkf\" (UniqueName: \"kubernetes.io/projected/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-kube-api-access-kswkf\") pod \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " Jul 29 12:58:19.560114 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.559997 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-plugins-config-volume\") pod \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\" (UID: \"3d0b33fb-1a08-48b3-b239-e1175bd36ff8\") " Jul 29 12:58:19.560292 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.560266 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13b5657a-a2f5-4e39-9899-7d2eee94c2e0" (UID: "13b5657a-a2f5-4e39-9899-7d2eee94c2e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:58:19.560412 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.560381 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d0b33fb-1a08-48b3-b239-e1175bd36ff8" (UID: "3d0b33fb-1a08-48b3-b239-e1175bd36ff8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:58:19.562004 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.561982 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-kube-api-access-kswkf" (OuterVolumeSpecName: "kube-api-access-kswkf") pod "3d0b33fb-1a08-48b3-b239-e1175bd36ff8" (UID: "3d0b33fb-1a08-48b3-b239-e1175bd36ff8"). InnerVolumeSpecName "kube-api-access-kswkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:19.562056 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.562029 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-kube-api-access-9flzp" (OuterVolumeSpecName: "kube-api-access-9flzp") pod "13b5657a-a2f5-4e39-9899-7d2eee94c2e0" (UID: "13b5657a-a2f5-4e39-9899-7d2eee94c2e0"). InnerVolumeSpecName "kube-api-access-9flzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:19.661010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.660921 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9flzp\" (UniqueName: \"kubernetes.io/projected/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-kube-api-access-9flzp\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:19.661010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.660955 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13b5657a-a2f5-4e39-9899-7d2eee94c2e0-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:19.661010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.660970 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kswkf\" (UniqueName: \"kubernetes.io/projected/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-kube-api-access-kswkf\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:19.661010 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.660984 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d0b33fb-1a08-48b3-b239-e1175bd36ff8-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:19.838928 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:19.838893 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:20.011260 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.011227 2563 generic.go:356] "Generic (PLEG): container finished" podID="13b5657a-a2f5-4e39-9899-7d2eee94c2e0" containerID="7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb" exitCode=0 Jul 29 12:58:20.011447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.011270 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" Jul 29 12:58:20.011447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.011302 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" event={"ID":"13b5657a-a2f5-4e39-9899-7d2eee94c2e0","Type":"ContainerDied","Data":"7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb"} Jul 29 12:58:20.011447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.011352 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l" event={"ID":"13b5657a-a2f5-4e39-9899-7d2eee94c2e0","Type":"ContainerDied","Data":"d3ae7847438da282165aeea3b02ea94a91961a178c05d7a5d0f30fd3578065a4"} Jul 29 12:58:20.011447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.011376 2563 scope.go:117] "RemoveContainer" containerID="7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb" Jul 29 12:58:20.012894 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.012857 2563 generic.go:356] "Generic (PLEG): container finished" podID="3d0b33fb-1a08-48b3-b239-e1175bd36ff8" containerID="3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5" exitCode=0 Jul 29 12:58:20.012894 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.012890 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" event={"ID":"3d0b33fb-1a08-48b3-b239-e1175bd36ff8","Type":"ContainerDied","Data":"3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5"} Jul 29 12:58:20.013034 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.012912 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" Jul 29 12:58:20.013034 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.012930 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z" event={"ID":"3d0b33fb-1a08-48b3-b239-e1175bd36ff8","Type":"ContainerDied","Data":"960091ef2850f20b34fd47ec1f199ee9f1fec3429652cd423e7f30f4702ae39f"} Jul 29 12:58:20.019880 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.019861 2563 scope.go:117] "RemoveContainer" containerID="7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb" Jul 29 12:58:20.020151 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:58:20.020133 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb\": container with ID starting with 7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb not found: ID does not exist" containerID="7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb" Jul 29 12:58:20.020199 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.020161 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb"} err="failed to get container status \"7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb\": rpc error: code = NotFound desc = could not find container \"7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb\": container with ID starting with 7a38362e484b6eb484e48708e3ac42d0ed3c314e4ec1b0eb5044703060fc52cb not found: ID does not exist" Jul 29 12:58:20.020199 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.020177 2563 scope.go:117] "RemoveContainer" containerID="3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5" Jul 29 12:58:20.028002 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.027967 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l"] Jul 29 12:58:20.028424 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.028384 2563 scope.go:117] "RemoveContainer" containerID="3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5" Jul 29 12:58:20.028742 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:58:20.028721 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5\": container with ID starting with 3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5 not found: ID does not exist" containerID="3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5" Jul 29 12:58:20.028808 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.028748 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5"} err="failed to get container status \"3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5\": rpc error: code = NotFound desc = could not find container \"3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5\": container with ID starting with 3af0bddc62158e713e0896a5fc1b10a64ebf65481a4d8e4a092b029e815e17b5 not found: ID does not exist" Jul 29 12:58:20.029900 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.029884 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-329c04f9-55f5458d8-hvs9l"] Jul 29 12:58:20.043236 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.043203 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z"] Jul 29 12:58:20.047436 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.047414 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-329c04f9-8965bf5dd-c645z"] Jul 29 12:58:20.518051 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.518017 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq"] Jul 29 12:58:20.518497 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.518217 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" containerID="cri-o://95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1" gracePeriod=30 Jul 29 12:58:20.542124 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.542093 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:20.778058 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.777956 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:20.995787 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:20.995753 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:21.278181 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278148 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg"] Jul 29 12:58:21.278731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278708 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" Jul 29 12:58:21.278731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278732 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278754 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278763 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278780 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b5657a-a2f5-4e39-9899-7d2eee94c2e0" containerName="payload-pre-processing" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278789 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b5657a-a2f5-4e39-9899-7d2eee94c2e0" containerName="payload-pre-processing" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278800 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d0b33fb-1a08-48b3-b239-e1175bd36ff8" containerName="payload-processing" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278808 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d0b33fb-1a08-48b3-b239-e1175bd36ff8" containerName="payload-processing" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278885 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278896 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b5657a-a2f5-4e39-9899-7d2eee94c2e0" containerName="payload-pre-processing" Jul 29 12:58:21.278913 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278907 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="343ada02-78fd-4133-8cc7-556a4715e5fc" containerName="istio-proxy" Jul 29 12:58:21.279198 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.278917 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d0b33fb-1a08-48b3-b239-e1175bd36ff8" containerName="payload-processing" Jul 29 12:58:21.280925 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.280908 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.283374 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.283352 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9210d382-openshift-default-dockercfg-q64f6\"" Jul 29 12:58:21.290872 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.290842 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg"] Jul 29 12:58:21.304144 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.304110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:21.374052 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-credential-socket\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374243 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374091 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-podinfo\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374243 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istiod-ca-cert\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374346 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-token\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374346 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-certs\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374412 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-data\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374447 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-envoy\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374531 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz2ht\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-kube-api-access-rz2ht\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.374531 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.374503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-socket\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475135 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-credential-socket\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475350 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475150 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-podinfo\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475350 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istiod-ca-cert\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475350 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-token\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475350 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475271 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-certs\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475638 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-data\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475638 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475591 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-envoy\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475638 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475618 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz2ht\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-kube-api-access-rz2ht\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475635 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-credential-socket\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475658 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-certs\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.475790 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.475661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-socket\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.476391 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.476191 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-data\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.476391 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.476327 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-socket\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.476391 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.476349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:21.476675 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.476650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istiod-ca-cert\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.477929 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.477909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-envoy\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.478140 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.478123 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-podinfo\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.483070 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.483048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-token\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.483155 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.483142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz2ht\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-kube-api-access-rz2ht\") pod \"e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.592684 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.592592 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:21.659948 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.659916 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b5657a-a2f5-4e39-9899-7d2eee94c2e0" path="/var/lib/kubelet/pods/13b5657a-a2f5-4e39-9899-7d2eee94c2e0/volumes" Jul 29 12:58:21.660263 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.660248 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d0b33fb-1a08-48b3-b239-e1175bd36ff8" path="/var/lib/kubelet/pods/3d0b33fb-1a08-48b3-b239-e1175bd36ff8/volumes" Jul 29 12:58:21.730870 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.730768 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg"] Jul 29 12:58:21.733434 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:58:21.733408 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0d22a0_fbc0_4b57_84be_96846fcd2bd8.slice/crio-c33f52a9f35248534b0c161e6c38809c4ac61ad2b0bd06a3dd108b7654652f13 WatchSource:0}: Error finding container c33f52a9f35248534b0c161e6c38809c4ac61ad2b0bd06a3dd108b7654652f13: Status 404 returned error can't find the container with id c33f52a9f35248534b0c161e6c38809c4ac61ad2b0bd06a3dd108b7654652f13 Jul 29 12:58:21.735612 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.735584 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:21.735680 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.735647 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:21.735680 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:21.735677 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:22.025731 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:22.025698 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" event={"ID":"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8","Type":"ContainerStarted","Data":"db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898"} Jul 29 12:58:22.025925 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:22.025750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" event={"ID":"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8","Type":"ContainerStarted","Data":"c33f52a9f35248534b0c161e6c38809c4ac61ad2b0bd06a3dd108b7654652f13"} Jul 29 12:58:22.047059 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:22.047004 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podStartSLOduration=1.046986013 podStartE2EDuration="1.046986013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:58:22.044825124 +0000 UTC m=+1974.948021811" watchObservedRunningTime="2026-07-29 12:58:22.046986013 +0000 UTC m=+1974.950182688" Jul 29 12:58:22.593122 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:22.593081 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:22.597898 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:22.597870 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:23.031315 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:23.031284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:23.032238 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:23.032221 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:23.400234 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:23.400142 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:23.477607 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:23.477564 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:23.560233 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:23.560195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:23.664044 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:23.663941 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:24.055147 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.055097 2563 patch_prober.go:28] interesting pod/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 12:58:24.055328 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.055184 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 12:58:24.199316 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.199282 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:24.299474 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.299434 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:24.303842 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.303809 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc"] Jul 29 12:58:24.306171 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.306122 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.308657 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.308636 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-9210d382-dockercfg-dzc7c\"" Jul 29 12:58:24.308772 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.308700 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-9210d382\"" Jul 29 12:58:24.320638 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.320614 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc"] Jul 29 12:58:24.342065 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.342035 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm"] Jul 29 12:58:24.344329 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.344303 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.354915 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.354892 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm"] Jul 29 12:58:24.405378 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.405339 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf561f3-ec78-406e-9f9b-0479ba7f4a20-plugins-config-volume\") pod \"payload-processing-e2e-derive-9210d382-8674d88778-x9xtm\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.405596 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.405420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0531ac59-84d9-4d49-b679-a6ffac043ad5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.405596 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.405497 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dcps\" (UniqueName: \"kubernetes.io/projected/abf561f3-ec78-406e-9f9b-0479ba7f4a20-kube-api-access-5dcps\") pod \"payload-processing-e2e-derive-9210d382-8674d88778-x9xtm\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.405596 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.405529 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2lg5\" (UniqueName: \"kubernetes.io/projected/0531ac59-84d9-4d49-b679-a6ffac043ad5-kube-api-access-f2lg5\") pod \"payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.506714 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.506674 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf561f3-ec78-406e-9f9b-0479ba7f4a20-plugins-config-volume\") pod \"payload-processing-e2e-derive-9210d382-8674d88778-x9xtm\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.506881 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.506729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0531ac59-84d9-4d49-b679-a6ffac043ad5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.506881 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.506774 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dcps\" (UniqueName: \"kubernetes.io/projected/abf561f3-ec78-406e-9f9b-0479ba7f4a20-kube-api-access-5dcps\") pod \"payload-processing-e2e-derive-9210d382-8674d88778-x9xtm\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.506881 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.506792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2lg5\" (UniqueName: \"kubernetes.io/projected/0531ac59-84d9-4d49-b679-a6ffac043ad5-kube-api-access-f2lg5\") pod \"payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.507473 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.507440 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf561f3-ec78-406e-9f9b-0479ba7f4a20-plugins-config-volume\") pod \"payload-processing-e2e-derive-9210d382-8674d88778-x9xtm\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.507537 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.507494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0531ac59-84d9-4d49-b679-a6ffac043ad5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.515398 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.515361 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dcps\" (UniqueName: \"kubernetes.io/projected/abf561f3-ec78-406e-9f9b-0479ba7f4a20-kube-api-access-5dcps\") pod \"payload-processing-e2e-derive-9210d382-8674d88778-x9xtm\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.515532 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.515435 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2lg5\" (UniqueName: \"kubernetes.io/projected/0531ac59-84d9-4d49-b679-a6ffac043ad5-kube-api-access-f2lg5\") pod \"payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.615977 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.615893 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:24.654531 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.654498 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:24.766148 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.765821 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc"] Jul 29 12:58:24.816623 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:24.816599 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm"] Jul 29 12:58:24.819667 ip-10-0-137-159 kubenswrapper[2563]: W0729 12:58:24.819643 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf561f3_ec78_406e_9f9b_0479ba7f4a20.slice/crio-df1c797bd400bfa95bbf3f2f5523611776762ea86f8647a0e0824dd6db0f508b WatchSource:0}: Error finding container df1c797bd400bfa95bbf3f2f5523611776762ea86f8647a0e0824dd6db0f508b: Status 404 returned error can't find the container with id df1c797bd400bfa95bbf3f2f5523611776762ea86f8647a0e0824dd6db0f508b Jul 29 12:58:25.041959 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.041925 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" event={"ID":"abf561f3-ec78-406e-9f9b-0479ba7f4a20","Type":"ContainerStarted","Data":"7a5136a44f9c6c61d1032e77da4b05bb0362ad4bfdeaac55ea43b7dfb6a2ca8d"} Jul 29 12:58:25.042149 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.042019 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" event={"ID":"abf561f3-ec78-406e-9f9b-0479ba7f4a20","Type":"ContainerStarted","Data":"df1c797bd400bfa95bbf3f2f5523611776762ea86f8647a0e0824dd6db0f508b"} Jul 29 12:58:25.042149 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.042051 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:25.043372 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.043342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" event={"ID":"0531ac59-84d9-4d49-b679-a6ffac043ad5","Type":"ContainerStarted","Data":"25016059c191c57d3620fbbe3aa826be049da1d7c75a0eb449a70151248c36f7"} Jul 29 12:58:25.043526 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.043394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" event={"ID":"0531ac59-84d9-4d49-b679-a6ffac043ad5","Type":"ContainerStarted","Data":"11668848cf310039178a6ab7c6267dff7bac4488697c46d8c0e85d206b634ec7"} Jul 29 12:58:25.059572 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.059532 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" podStartSLOduration=1.059518933 podStartE2EDuration="1.059518933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:58:25.057640485 +0000 UTC m=+1977.960837162" watchObservedRunningTime="2026-07-29 12:58:25.059518933 +0000 UTC m=+1977.962715609" Jul 29 12:58:25.074685 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.074644 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" podStartSLOduration=1.07463154 podStartE2EDuration="1.07463154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 12:58:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 12:58:25.072614311 +0000 UTC m=+1977.975810987" watchObservedRunningTime="2026-07-29 12:58:25.07463154 +0000 UTC m=+1977.977828216" Jul 29 12:58:25.786794 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.786742 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:58:25.920932 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.920906 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2eca81-a710-4509-9cb6-b4d86720743c-istiod-ca-cert\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.920948 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-socket\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.920979 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-data\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921022 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-envoy\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921053 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-podinfo\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921085 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921082 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-token\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921104 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-credential-socket\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921139 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vx54k\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-kube-api-access-vx54k\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921165 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-certs\") pod \"2e2eca81-a710-4509-9cb6-b4d86720743c\" (UID: \"2e2eca81-a710-4509-9cb6-b4d86720743c\") " Jul 29 12:58:25.921337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921215 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:25.921337 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921262 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2eca81-a710-4509-9cb6-b4d86720743c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:58:25.921618 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921448 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:25.921618 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921481 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:25.921618 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921530 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2eca81-a710-4509-9cb6-b4d86720743c-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:25.921618 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921549 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:25.921815 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.921756 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-data" (OuterVolumeSpecName: "istio-data") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:25.923144 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.923112 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 12:58:25.923267 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.923250 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-kube-api-access-vx54k" (OuterVolumeSpecName: "kube-api-access-vx54k") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "kube-api-access-vx54k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:25.923561 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.923536 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:25.923643 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:25.923543 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-token" (OuterVolumeSpecName: "istio-token") pod "2e2eca81-a710-4509-9cb6-b4d86720743c" (UID: "2e2eca81-a710-4509-9cb6-b4d86720743c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:26.022295 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022272 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.022380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022297 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.022380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022312 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.022380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022320 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.022380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022328 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.022380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022336 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vx54k\" (UniqueName: \"kubernetes.io/projected/2e2eca81-a710-4509-9cb6-b4d86720743c-kube-api-access-vx54k\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.022380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.022346 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2eca81-a710-4509-9cb6-b4d86720743c-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:26.054220 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.054149 2563 generic.go:356] "Generic (PLEG): container finished" podID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerID="95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1" exitCode=0 Jul 29 12:58:26.054220 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.054207 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" Jul 29 12:58:26.054347 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.054237 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" event={"ID":"2e2eca81-a710-4509-9cb6-b4d86720743c","Type":"ContainerDied","Data":"95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1"} Jul 29 12:58:26.054347 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.054287 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq" event={"ID":"2e2eca81-a710-4509-9cb6-b4d86720743c","Type":"ContainerDied","Data":"5c24cb901d54776deb01ad15b7ddcae53c4cf6e38253585894fa7d6e9efe6bfa"} Jul 29 12:58:26.054347 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.054302 2563 scope.go:117] "RemoveContainer" containerID="95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1" Jul 29 12:58:26.054621 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.054588 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:26.063516 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.063498 2563 scope.go:117] "RemoveContainer" containerID="92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1" Jul 29 12:58:26.071095 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.071073 2563 scope.go:117] "RemoveContainer" containerID="95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1" Jul 29 12:58:26.071420 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:58:26.071401 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1\": container with ID starting with 95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1 not found: ID does not exist" containerID="95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1" Jul 29 12:58:26.071721 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.071430 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1"} err="failed to get container status \"95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1\": rpc error: code = NotFound desc = could not find container \"95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1\": container with ID starting with 95c52cb313650da804497cb57e0decc2f8733aa70d832214ed66bccf07cf51f1 not found: ID does not exist" Jul 29 12:58:26.071721 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.071574 2563 scope.go:117] "RemoveContainer" containerID="92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1" Jul 29 12:58:26.071860 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:58:26.071842 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1\": container with ID starting with 92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1 not found: ID does not exist" containerID="92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1" Jul 29 12:58:26.071917 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.071865 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1"} err="failed to get container status \"92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1\": rpc error: code = NotFound desc = could not find container \"92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1\": container with ID starting with 92bbdf457940c25e4120224add7591386dedce40f8a4cb204c507862867307a1 not found: ID does not exist" Jul 29 12:58:26.075973 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.075951 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq"] Jul 29 12:58:26.080188 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:26.080168 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-329c04f9-openshift-default-cfc949996-jnhkq"] Jul 29 12:58:27.658572 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:27.658540 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" path="/var/lib/kubelet/pods/2e2eca81-a710-4509-9cb6-b4d86720743c/volumes" Jul 29 12:58:36.056884 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:36.056851 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:37.062904 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.062865 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" podUID="0531ac59-84d9-4d49-b679-a6ffac043ad5" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.44:9004: connect: connection refused" Jul 29 12:58:37.112021 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.111994 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerID="db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898" exitCode=0 Jul 29 12:58:37.112113 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.112039 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" event={"ID":"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8","Type":"ContainerDied","Data":"db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898"} Jul 29 12:58:37.112407 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.112395 2563 scope.go:117] "RemoveContainer" containerID="db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898" Jul 29 12:58:37.113383 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.113352 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:37.113438 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.113422 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:37.113496 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:37.113455 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 12:58:38.031427 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:38.031395 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:38.117948 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:38.117909 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" event={"ID":"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8","Type":"ContainerStarted","Data":"8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf"} Jul 29 12:58:38.593653 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:38.593617 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:38.594771 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:38.594749 2563 patch_prober.go:28] interesting pod/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 29 12:58:38.594880 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:38.594790 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 29 12:58:39.593056 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:39.593015 2563 patch_prober.go:28] interesting pod/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 29 12:58:39.593432 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:39.593079 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 29 12:58:40.593476 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:40.593418 2563 patch_prober.go:28] interesting pod/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 29 12:58:40.593925 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:40.593504 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 29 12:58:41.593418 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:41.593380 2563 patch_prober.go:28] interesting pod/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 29 12:58:41.593608 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:41.593444 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 29 12:58:42.597022 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:42.596994 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:42.597379 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:42.597353 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:42.598151 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:42.598134 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:47.061580 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:47.061547 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:54.037081 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.037045 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:54.062671 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.062618 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm"] Jul 29 12:58:54.063394 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.063363 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" podUID="abf561f3-ec78-406e-9f9b-0479ba7f4a20" containerName="payload-processing" containerID="cri-o://7a5136a44f9c6c61d1032e77da4b05bb0362ad4bfdeaac55ea43b7dfb6a2ca8d" gracePeriod=30 Jul 29 12:58:54.071269 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.071244 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc"] Jul 29 12:58:54.071771 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.071591 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" podUID="0531ac59-84d9-4d49-b679-a6ffac043ad5" containerName="payload-pre-processing" containerID="cri-o://25016059c191c57d3620fbbe3aa826be049da1d7c75a0eb449a70151248c36f7" gracePeriod=30 Jul 29 12:58:54.105965 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.105939 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:54.197833 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.197700 2563 generic.go:356] "Generic (PLEG): container finished" podID="abf561f3-ec78-406e-9f9b-0479ba7f4a20" containerID="7a5136a44f9c6c61d1032e77da4b05bb0362ad4bfdeaac55ea43b7dfb6a2ca8d" exitCode=0 Jul 29 12:58:54.197833 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.197766 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" event={"ID":"abf561f3-ec78-406e-9f9b-0479ba7f4a20","Type":"ContainerDied","Data":"7a5136a44f9c6c61d1032e77da4b05bb0362ad4bfdeaac55ea43b7dfb6a2ca8d"} Jul 29 12:58:54.199311 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.199290 2563 generic.go:356] "Generic (PLEG): container finished" podID="0531ac59-84d9-4d49-b679-a6ffac043ad5" containerID="25016059c191c57d3620fbbe3aa826be049da1d7c75a0eb449a70151248c36f7" exitCode=0 Jul 29 12:58:54.199406 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.199360 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" event={"ID":"0531ac59-84d9-4d49-b679-a6ffac043ad5","Type":"ContainerDied","Data":"25016059c191c57d3620fbbe3aa826be049da1d7c75a0eb449a70151248c36f7"} Jul 29 12:58:54.326058 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.326026 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:54.335555 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.335530 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg"] Jul 29 12:58:54.335755 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.335733 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" containerID="cri-o://8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf" gracePeriod=30 Jul 29 12:58:54.368692 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.368618 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:54.467293 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.467258 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf561f3-ec78-406e-9f9b-0479ba7f4a20-plugins-config-volume\") pod \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " Jul 29 12:58:54.467483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.467323 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2lg5\" (UniqueName: \"kubernetes.io/projected/0531ac59-84d9-4d49-b679-a6ffac043ad5-kube-api-access-f2lg5\") pod \"0531ac59-84d9-4d49-b679-a6ffac043ad5\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " Jul 29 12:58:54.467483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.467360 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dcps\" (UniqueName: \"kubernetes.io/projected/abf561f3-ec78-406e-9f9b-0479ba7f4a20-kube-api-access-5dcps\") pod \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\" (UID: \"abf561f3-ec78-406e-9f9b-0479ba7f4a20\") " Jul 29 12:58:54.467483 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.467387 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0531ac59-84d9-4d49-b679-a6ffac043ad5-plugins-config-volume\") pod \"0531ac59-84d9-4d49-b679-a6ffac043ad5\" (UID: \"0531ac59-84d9-4d49-b679-a6ffac043ad5\") " Jul 29 12:58:54.467794 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.467770 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf561f3-ec78-406e-9f9b-0479ba7f4a20-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abf561f3-ec78-406e-9f9b-0479ba7f4a20" (UID: "abf561f3-ec78-406e-9f9b-0479ba7f4a20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:58:54.467881 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.467830 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0531ac59-84d9-4d49-b679-a6ffac043ad5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0531ac59-84d9-4d49-b679-a6ffac043ad5" (UID: "0531ac59-84d9-4d49-b679-a6ffac043ad5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:58:54.469524 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.469494 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf561f3-ec78-406e-9f9b-0479ba7f4a20-kube-api-access-5dcps" (OuterVolumeSpecName: "kube-api-access-5dcps") pod "abf561f3-ec78-406e-9f9b-0479ba7f4a20" (UID: "abf561f3-ec78-406e-9f9b-0479ba7f4a20"). InnerVolumeSpecName "kube-api-access-5dcps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:54.469897 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.469881 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0531ac59-84d9-4d49-b679-a6ffac043ad5-kube-api-access-f2lg5" (OuterVolumeSpecName: "kube-api-access-f2lg5") pod "0531ac59-84d9-4d49-b679-a6ffac043ad5" (UID: "0531ac59-84d9-4d49-b679-a6ffac043ad5"). InnerVolumeSpecName "kube-api-access-f2lg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:54.568993 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.568919 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2lg5\" (UniqueName: \"kubernetes.io/projected/0531ac59-84d9-4d49-b679-a6ffac043ad5-kube-api-access-f2lg5\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:54.568993 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.568951 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dcps\" (UniqueName: \"kubernetes.io/projected/abf561f3-ec78-406e-9f9b-0479ba7f4a20-kube-api-access-5dcps\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:54.568993 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.568965 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0531ac59-84d9-4d49-b679-a6ffac043ad5-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:54.568993 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.568979 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf561f3-ec78-406e-9f9b-0479ba7f4a20-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:54.676003 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.675967 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:54.736445 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:54.736415 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:55.062826 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.062792 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:55.161133 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.161100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:55.205709 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.205682 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" Jul 29 12:58:55.205872 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.205706 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm" event={"ID":"abf561f3-ec78-406e-9f9b-0479ba7f4a20","Type":"ContainerDied","Data":"df1c797bd400bfa95bbf3f2f5523611776762ea86f8647a0e0824dd6db0f508b"} Jul 29 12:58:55.205872 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.205799 2563 scope.go:117] "RemoveContainer" containerID="7a5136a44f9c6c61d1032e77da4b05bb0362ad4bfdeaac55ea43b7dfb6a2ca8d" Jul 29 12:58:55.207115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.207080 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" event={"ID":"0531ac59-84d9-4d49-b679-a6ffac043ad5","Type":"ContainerDied","Data":"11668848cf310039178a6ab7c6267dff7bac4488697c46d8c0e85d206b634ec7"} Jul 29 12:58:55.207115 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.207101 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc" Jul 29 12:58:55.215318 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.215303 2563 scope.go:117] "RemoveContainer" containerID="25016059c191c57d3620fbbe3aa826be049da1d7c75a0eb449a70151248c36f7" Jul 29 12:58:55.230534 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.230512 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm"] Jul 29 12:58:55.235344 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.235323 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-9210d382-8674d88778-x9xtm"] Jul 29 12:58:55.238696 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.238676 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:58:55.248486 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.248449 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc"] Jul 29 12:58:55.252748 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.252729 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-9210d382-5fcb989968-szzwc"] Jul 29 12:58:55.658792 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.658760 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0531ac59-84d9-4d49-b679-a6ffac043ad5" path="/var/lib/kubelet/pods/0531ac59-84d9-4d49-b679-a6ffac043ad5/volumes" Jul 29 12:58:55.659146 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:55.659132 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf561f3-ec78-406e-9f9b-0479ba7f4a20" path="/var/lib/kubelet/pods/abf561f3-ec78-406e-9f9b-0479ba7f4a20/volumes" Jul 29 12:58:57.599368 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:57.599327 2563 patch_prober.go:28] interesting pod/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 12:58:57.599742 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:57.599397 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 12:58:59.582380 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.582359 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:58:59.603501 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603287 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz2ht\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-kube-api-access-rz2ht\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603501 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603338 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-token\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603501 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603376 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-certs\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603501 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603428 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-podinfo\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603501 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603451 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-envoy\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603844 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603507 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istiod-ca-cert\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603844 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603529 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-socket\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603844 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603560 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-credential-socket\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.603844 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.603626 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-data\") pod \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\" (UID: \"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8\") " Jul 29 12:58:59.604599 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.604344 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 12:58:59.605207 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.604903 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:59.605995 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.605970 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:59.606557 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.606530 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:59.608042 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.607988 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:59.608142 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.608111 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:59.608452 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.608369 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 12:58:59.608452 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.608393 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-kube-api-access-rz2ht" (OuterVolumeSpecName: "kube-api-access-rz2ht") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "kube-api-access-rz2ht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 12:58:59.608600 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.608519 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" (UID: "5f0d22a0-fbc0-4b57-84be-96846fcd2bd8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 12:58:59.704318 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704291 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz2ht\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-kube-api-access-rz2ht\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704318 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704316 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704327 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704336 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704344 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704353 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704361 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704369 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:58:59.704553 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:58:59.704376 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 12:59:00.233734 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.233696 2563 generic.go:356] "Generic (PLEG): container finished" podID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerID="8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf" exitCode=0 Jul 29 12:59:00.233871 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.233740 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" Jul 29 12:59:00.233871 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.233774 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" event={"ID":"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8","Type":"ContainerDied","Data":"8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf"} Jul 29 12:59:00.233871 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.233832 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg" event={"ID":"5f0d22a0-fbc0-4b57-84be-96846fcd2bd8","Type":"ContainerDied","Data":"c33f52a9f35248534b0c161e6c38809c4ac61ad2b0bd06a3dd108b7654652f13"} Jul 29 12:59:00.233871 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.233861 2563 scope.go:117] "RemoveContainer" containerID="8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf" Jul 29 12:59:00.244505 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.244452 2563 scope.go:117] "RemoveContainer" containerID="db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898" Jul 29 12:59:00.252114 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.252089 2563 scope.go:117] "RemoveContainer" containerID="8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf" Jul 29 12:59:00.252413 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:59:00.252395 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf\": container with ID starting with 8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf not found: ID does not exist" containerID="8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf" Jul 29 12:59:00.252492 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.252420 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf"} err="failed to get container status \"8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf\": rpc error: code = NotFound desc = could not find container \"8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf\": container with ID starting with 8d2d7b1ab76a35c5dc1acfa8c7cf4af7d52062ea84415a319d93a6b9781db0bf not found: ID does not exist" Jul 29 12:59:00.252492 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.252449 2563 scope.go:117] "RemoveContainer" containerID="db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898" Jul 29 12:59:00.252690 ip-10-0-137-159 kubenswrapper[2563]: E0729 12:59:00.252676 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898\": container with ID starting with db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898 not found: ID does not exist" containerID="db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898" Jul 29 12:59:00.252750 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.252695 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898"} err="failed to get container status \"db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898\": rpc error: code = NotFound desc = could not find container \"db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898\": container with ID starting with db704cec1a807e0dda177b9a37928b5defbab3619ce85fc195633ec776883898 not found: ID does not exist" Jul 29 12:59:00.253485 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.253451 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg"] Jul 29 12:59:00.258867 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:00.258842 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9210d382-openshift-default-6db7d46c48-5wdxg"] Jul 29 12:59:01.657918 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:01.657884 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" path="/var/lib/kubelet/pods/5f0d22a0-fbc0-4b57-84be-96846fcd2bd8/volumes" Jul 29 12:59:17.227208 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:17.227166 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:24.592947 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:24.592916 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:24.710884 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:24.710855 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:24.901548 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:24.901438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:25.003622 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:25.003588 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:25.366164 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:25.366128 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:51.329684 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:51.329605 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:57.941047 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:57.941013 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:58.048939 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:58.048908 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 12:59:58.141158 ip-10-0-137-159 kubenswrapper[2563]: I0729 12:59:58.141124 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:14.064850 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:14.064810 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:20.119018 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:20.118981 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:20.218405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:20.218365 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:30.629430 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:30.629390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:41.706681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:41.706641 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:48.022388 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:48.022354 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:48.129432 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:48.129388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:48.222437 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:48.222402 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:00:48.323259 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:00:48.323161 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:01:19.414850 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:01:19.414757 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:01:30.699782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:01:30.699749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:01:34.760394 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:01:34.760346 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:50.519530 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:50.519422 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:57.582012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:57.581957 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:57.682577 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:57.682545 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:58.361681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.361649 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:58.646183 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646107 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh"] Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646438 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf561f3-ec78-406e-9f9b-0479ba7f4a20" containerName="payload-processing" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646451 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf561f3-ec78-406e-9f9b-0479ba7f4a20" containerName="payload-processing" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646487 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0531ac59-84d9-4d49-b679-a6ffac043ad5" containerName="payload-pre-processing" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646494 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0531ac59-84d9-4d49-b679-a6ffac043ad5" containerName="payload-pre-processing" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646507 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646514 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646525 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646530 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646538 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" Jul 29 13:02:58.646542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646544 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646558 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646566 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646616 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646623 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf561f3-ec78-406e-9f9b-0479ba7f4a20" containerName="payload-processing" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646634 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646643 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f0d22a0-fbc0-4b57-84be-96846fcd2bd8" containerName="istio-proxy" Jul 29 13:02:58.646864 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.646650 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0531ac59-84d9-4d49-b679-a6ffac043ad5" containerName="payload-pre-processing" Jul 29 13:02:58.649565 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.649544 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.652477 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.652445 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-be3b8a53-openshift-default-dockercfg-pd629\"" Jul 29 13:02:58.674201 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.674176 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh"] Jul 29 13:02:58.675690 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.675662 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:58.724508 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-socket\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724621 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-token\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724621 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724585 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-certs\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724699 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724621 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-podinfo\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724699 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724644 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-envoy\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724699 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724663 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmjs\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-kube-api-access-9zmjs\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724793 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-credential-socket\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724838 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-data\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.724874 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.724846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274b53fe-9fb2-4a7a-a3a8-681930651af9-istiod-ca-cert\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826080 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826044 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-credential-socket\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826228 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826089 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-data\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826228 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826106 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274b53fe-9fb2-4a7a-a3a8-681930651af9-istiod-ca-cert\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826322 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826227 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-socket\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826322 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-token\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826411 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-certs\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826411 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826347 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-podinfo\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826541 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826513 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-credential-socket\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826594 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-data\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826646 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826628 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-socket\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826699 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-certs\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826748 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-envoy\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826748 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826732 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmjs\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-kube-api-access-9zmjs\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.826848 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.826827 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274b53fe-9fb2-4a7a-a3a8-681930651af9-istiod-ca-cert\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.828731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.828701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-envoy\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.828907 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.828891 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-podinfo\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.833765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.833734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-token\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.833880 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.833847 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmjs\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-kube-api-access-9zmjs\") pod \"e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:58.960153 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:58.960133 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:02:59.017398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.017371 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:02:59.100300 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.100277 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh"] Jul 29 13:02:59.102432 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:02:59.102405 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274b53fe_9fb2_4a7a_a3a8_681930651af9.slice/crio-46ed0b8a25c0531e7ce46b2823b820b7925bea470e45a83c8037da40b8f87974 WatchSource:0}: Error finding container 46ed0b8a25c0531e7ce46b2823b820b7925bea470e45a83c8037da40b8f87974: Status 404 returned error can't find the container with id 46ed0b8a25c0531e7ce46b2823b820b7925bea470e45a83c8037da40b8f87974 Jul 29 13:02:59.104522 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.104493 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:02:59.104586 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.104552 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:02:59.104586 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.104581 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:02:59.283834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.283741 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" event={"ID":"274b53fe-9fb2-4a7a-a3a8-681930651af9","Type":"ContainerStarted","Data":"6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd"} Jul 29 13:02:59.283834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.283803 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" event={"ID":"274b53fe-9fb2-4a7a-a3a8-681930651af9","Type":"ContainerStarted","Data":"46ed0b8a25c0531e7ce46b2823b820b7925bea470e45a83c8037da40b8f87974"} Jul 29 13:02:59.307094 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.307044 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podStartSLOduration=1.307029225 podStartE2EDuration="1.307029225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:02:59.306350733 +0000 UTC m=+2252.209547411" watchObservedRunningTime="2026-07-29 13:02:59.307029225 +0000 UTC m=+2252.210225902" Jul 29 13:02:59.961111 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:02:59.961070 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:00.966765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:00.966734 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:01.292860 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:01.292784 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:01.293613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:01.293588 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:01.400507 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:01.400430 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:01.503435 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:01.503388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:01.583522 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:01.583393 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:02.468717 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.468680 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5"] Jul 29 13:03:02.469257 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.469236 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e2eca81-a710-4509-9cb6-b4d86720743c" containerName="istio-proxy" Jul 29 13:03:02.472105 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.472083 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.475354 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.475336 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-be3b8a53-dockercfg-mgmzf\"" Jul 29 13:03:02.475701 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.475686 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-be3b8a53\"" Jul 29 13:03:02.501288 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.501260 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5"] Jul 29 13:03:02.560362 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.560332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v59j\" (UniqueName: \"kubernetes.io/projected/c698a358-36be-4f4f-995f-2b679bb733b3-kube-api-access-9v59j\") pod \"payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.560551 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.560374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c698a358-36be-4f4f-995f-2b679bb733b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.589793 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.589757 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q"] Jul 29 13:03:02.593303 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.593283 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.606096 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.606076 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q"] Jul 29 13:03:02.661016 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.660985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwdzd\" (UniqueName: \"kubernetes.io/projected/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-kube-api-access-dwdzd\") pod \"payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.661162 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.661128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v59j\" (UniqueName: \"kubernetes.io/projected/c698a358-36be-4f4f-995f-2b679bb733b3-kube-api-access-9v59j\") pod \"payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.661230 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.661172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c698a358-36be-4f4f-995f-2b679bb733b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.661230 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.661202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-plugins-config-volume\") pod \"payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.662356 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.662332 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c698a358-36be-4f4f-995f-2b679bb733b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.671221 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.671198 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v59j\" (UniqueName: \"kubernetes.io/projected/c698a358-36be-4f4f-995f-2b679bb733b3-kube-api-access-9v59j\") pod \"payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.762261 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.762193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-plugins-config-volume\") pod \"payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.762383 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.762268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwdzd\" (UniqueName: \"kubernetes.io/projected/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-kube-api-access-dwdzd\") pod \"payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.763084 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.763042 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-plugins-config-volume\") pod \"payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.771050 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.771024 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwdzd\" (UniqueName: \"kubernetes.io/projected/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-kube-api-access-dwdzd\") pod \"payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.782405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.782386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:02.904267 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.904235 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:02.915388 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:02.915352 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5"] Jul 29 13:03:02.916705 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:03:02.916673 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc698a358_36be_4f4f_995f_2b679bb733b3.slice/crio-fd852110883b508e156e8332dc266ba72e2397b4c431a6105a25e07f558f366d WatchSource:0}: Error finding container fd852110883b508e156e8332dc266ba72e2397b4c431a6105a25e07f558f366d: Status 404 returned error can't find the container with id fd852110883b508e156e8332dc266ba72e2397b4c431a6105a25e07f558f366d Jul 29 13:03:03.249586 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.249408 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q"] Jul 29 13:03:03.251770 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:03:03.251742 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a606b9_85d3_4c54_8cec_a3510c8c1bcc.slice/crio-5d155725d2c7322f50c80ef8cc6b4a66d82a7c20e7c42721c70f2dc5deaa5d31 WatchSource:0}: Error finding container 5d155725d2c7322f50c80ef8cc6b4a66d82a7c20e7c42721c70f2dc5deaa5d31: Status 404 returned error can't find the container with id 5d155725d2c7322f50c80ef8cc6b4a66d82a7c20e7c42721c70f2dc5deaa5d31 Jul 29 13:03:03.303428 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.303394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" event={"ID":"c698a358-36be-4f4f-995f-2b679bb733b3","Type":"ContainerStarted","Data":"0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c"} Jul 29 13:03:03.303553 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.303482 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:03.303553 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.303507 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" event={"ID":"c698a358-36be-4f4f-995f-2b679bb733b3","Type":"ContainerStarted","Data":"fd852110883b508e156e8332dc266ba72e2397b4c431a6105a25e07f558f366d"} Jul 29 13:03:03.304561 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.304540 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" event={"ID":"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc","Type":"ContainerStarted","Data":"5d155725d2c7322f50c80ef8cc6b4a66d82a7c20e7c42721c70f2dc5deaa5d31"} Jul 29 13:03:03.320933 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.320860 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" podStartSLOduration=1.320841615 podStartE2EDuration="1.320841615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:03:03.319436296 +0000 UTC m=+2256.222632974" watchObservedRunningTime="2026-07-29 13:03:03.320841615 +0000 UTC m=+2256.224038296" Jul 29 13:03:03.990543 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:03.990499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:04.090598 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:04.090568 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:04.310934 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:04.310853 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" event={"ID":"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc","Type":"ContainerStarted","Data":"4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260"} Jul 29 13:03:04.327794 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:04.327741 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" podStartSLOduration=2.32772286 podStartE2EDuration="2.32772286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:03:04.327515098 +0000 UTC m=+2257.230711775" watchObservedRunningTime="2026-07-29 13:03:04.32772286 +0000 UTC m=+2257.230919537" Jul 29 13:03:05.315731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:05.315700 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:07.259415 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:07.259383 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:07.333916 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:07.333876 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:14.312828 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:14.312782 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.47:9004: connect: connection refused" Jul 29 13:03:15.361917 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:15.361815 2563 generic.go:356] "Generic (PLEG): container finished" podID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerID="6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd" exitCode=0 Jul 29 13:03:15.361917 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:15.361844 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" event={"ID":"274b53fe-9fb2-4a7a-a3a8-681930651af9","Type":"ContainerDied","Data":"6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd"} Jul 29 13:03:15.362511 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:15.362272 2563 scope.go:117] "RemoveContainer" containerID="6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd" Jul 29 13:03:15.363418 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:15.363388 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:03:15.363531 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:15.363489 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:03:15.363598 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:15.363528 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:03:16.292795 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:16.292763 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:16.322811 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:16.322783 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:16.379305 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:16.379268 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" event={"ID":"274b53fe-9fb2-4a7a-a3a8-681930651af9","Type":"ContainerStarted","Data":"506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a"} Jul 29 13:03:16.961626 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:16.961579 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:16.962002 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:16.961972 2563 patch_prober.go:28] interesting pod/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 29 13:03:16.962075 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:16.962052 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 29 13:03:17.961181 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:17.961136 2563 patch_prober.go:28] interesting pod/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 29 13:03:17.961722 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:17.961204 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 29 13:03:18.961341 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:18.961293 2563 patch_prober.go:28] interesting pod/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 29 13:03:18.961907 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:18.961374 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 29 13:03:19.961453 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:19.961409 2563 patch_prober.go:28] interesting pod/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 29 13:03:19.962110 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:19.962070 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 29 13:03:20.964903 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:20.964872 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:20.965308 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:20.965138 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:20.965879 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:20.965864 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:24.312637 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:24.312606 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:26.434700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:26.434652 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:27.112751 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:27.112693 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:27.212342 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:27.212307 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:27.590812 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:27.590777 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:33.184609 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:33.184573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:33.289961 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:33.289915 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:44.083744 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.083686 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:44.112170 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.112139 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q"] Jul 29 13:03:44.112407 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.112381 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" podUID="c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" containerName="payload-processing" containerID="cri-o://4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260" gracePeriod=30 Jul 29 13:03:44.122728 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.122699 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5"] Jul 29 13:03:44.124039 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.123061 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" containerName="payload-pre-processing" containerID="cri-o://0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c" gracePeriod=30 Jul 29 13:03:44.183961 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.183924 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:44.256697 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.256660 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:44.312129 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.312082 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.47:9004: connect: connection refused" Jul 29 13:03:44.322998 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.322940 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:44.384220 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.384195 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:44.409102 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.409072 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:44.441204 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.441177 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-plugins-config-volume\") pod \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " Jul 29 13:03:44.441361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.441228 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwdzd\" (UniqueName: \"kubernetes.io/projected/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-kube-api-access-dwdzd\") pod \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\" (UID: \"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc\") " Jul 29 13:03:44.441361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.441269 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c698a358-36be-4f4f-995f-2b679bb733b3-plugins-config-volume\") pod \"c698a358-36be-4f4f-995f-2b679bb733b3\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " Jul 29 13:03:44.441361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.441295 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v59j\" (UniqueName: \"kubernetes.io/projected/c698a358-36be-4f4f-995f-2b679bb733b3-kube-api-access-9v59j\") pod \"c698a358-36be-4f4f-995f-2b679bb733b3\" (UID: \"c698a358-36be-4f4f-995f-2b679bb733b3\") " Jul 29 13:03:44.441648 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.441616 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" (UID: "c6a606b9-85d3-4c54-8cec-a3510c8c1bcc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:03:44.441753 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.441688 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c698a358-36be-4f4f-995f-2b679bb733b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c698a358-36be-4f4f-995f-2b679bb733b3" (UID: "c698a358-36be-4f4f-995f-2b679bb733b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:03:44.443471 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.443429 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c698a358-36be-4f4f-995f-2b679bb733b3-kube-api-access-9v59j" (OuterVolumeSpecName: "kube-api-access-9v59j") pod "c698a358-36be-4f4f-995f-2b679bb733b3" (UID: "c698a358-36be-4f4f-995f-2b679bb733b3"). InnerVolumeSpecName "kube-api-access-9v59j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:03:44.443566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.443488 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-kube-api-access-dwdzd" (OuterVolumeSpecName: "kube-api-access-dwdzd") pod "c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" (UID: "c6a606b9-85d3-4c54-8cec-a3510c8c1bcc"). InnerVolumeSpecName "kube-api-access-dwdzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:03:44.510846 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.510809 2563 generic.go:356] "Generic (PLEG): container finished" podID="c698a358-36be-4f4f-995f-2b679bb733b3" containerID="0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c" exitCode=0 Jul 29 13:03:44.511044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.510850 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" Jul 29 13:03:44.511044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.510892 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" event={"ID":"c698a358-36be-4f4f-995f-2b679bb733b3","Type":"ContainerDied","Data":"0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c"} Jul 29 13:03:44.511044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.510969 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5" event={"ID":"c698a358-36be-4f4f-995f-2b679bb733b3","Type":"ContainerDied","Data":"fd852110883b508e156e8332dc266ba72e2397b4c431a6105a25e07f558f366d"} Jul 29 13:03:44.511044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.510995 2563 scope.go:117] "RemoveContainer" containerID="0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c" Jul 29 13:03:44.512298 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.512275 2563 generic.go:356] "Generic (PLEG): container finished" podID="c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" containerID="4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260" exitCode=0 Jul 29 13:03:44.512404 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.512322 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" Jul 29 13:03:44.512404 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.512318 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" event={"ID":"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc","Type":"ContainerDied","Data":"4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260"} Jul 29 13:03:44.512535 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.512410 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q" event={"ID":"c6a606b9-85d3-4c54-8cec-a3510c8c1bcc","Type":"ContainerDied","Data":"5d155725d2c7322f50c80ef8cc6b4a66d82a7c20e7c42721c70f2dc5deaa5d31"} Jul 29 13:03:44.520047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.519923 2563 scope.go:117] "RemoveContainer" containerID="0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c" Jul 29 13:03:44.520267 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:03:44.520211 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c\": container with ID starting with 0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c not found: ID does not exist" containerID="0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c" Jul 29 13:03:44.520267 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.520243 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c"} err="failed to get container status \"0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c\": rpc error: code = NotFound desc = could not find container \"0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c\": container with ID starting with 0eb741e8587ea13191524985e3bb7912cceeb3584c2d2b3b71a431a2fa79489c not found: ID does not exist" Jul 29 13:03:44.520267 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.520265 2563 scope.go:117] "RemoveContainer" containerID="4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260" Jul 29 13:03:44.527797 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.527782 2563 scope.go:117] "RemoveContainer" containerID="4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260" Jul 29 13:03:44.528031 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:03:44.528014 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260\": container with ID starting with 4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260 not found: ID does not exist" containerID="4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260" Jul 29 13:03:44.528076 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.528036 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260"} err="failed to get container status \"4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260\": rpc error: code = NotFound desc = could not find container \"4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260\": container with ID starting with 4418755551e87e99b5be6c8e334f7f272e2851eacd3ebbedda2f0c823a24b260 not found: ID does not exist" Jul 29 13:03:44.535284 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.535263 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q"] Jul 29 13:03:44.538493 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.538473 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-be3b8a53-7586695d98-qf52q"] Jul 29 13:03:44.541925 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.541908 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwdzd\" (UniqueName: \"kubernetes.io/projected/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-kube-api-access-dwdzd\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:44.541985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.541928 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c698a358-36be-4f4f-995f-2b679bb733b3-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:44.541985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.541939 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v59j\" (UniqueName: \"kubernetes.io/projected/c698a358-36be-4f4f-995f-2b679bb733b3-kube-api-access-9v59j\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:44.541985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.541967 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:44.548273 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.548253 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5"] Jul 29 13:03:44.552030 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.552011 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-be3b8a53-559574574d-nwlp5"] Jul 29 13:03:44.932628 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:44.932587 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:45.659424 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:45.659386 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" path="/var/lib/kubelet/pods/c698a358-36be-4f4f-995f-2b679bb733b3/volumes" Jul 29 13:03:45.659835 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:45.659819 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" path="/var/lib/kubelet/pods/c6a606b9-85d3-4c54-8cec-a3510c8c1bcc/volumes" Jul 29 13:03:50.612788 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.612751 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:50.681287 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.681250 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:50.868120 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.868036 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh"] Jul 29 13:03:50.868359 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.868304 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" containerID="cri-o://506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a" gracePeriod=30 Jul 29 13:03:50.884130 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.884099 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:50.968111 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.968068 2563 patch_prober.go:28] interesting pod/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 13:03:50.968328 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:50.968159 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 13:03:51.137809 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:51.137720 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:51.261699 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:51.261658 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:51.361657 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:51.361621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:51.492828 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:51.492789 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:56.121096 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.121072 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:56.256781 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256748 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zmjs\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-kube-api-access-9zmjs\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256796 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-socket\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256823 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-envoy\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256853 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-certs\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256868 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274b53fe-9fb2-4a7a-a3a8-681930651af9-istiod-ca-cert\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256906 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-podinfo\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256946 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-data\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.256972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256975 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-token\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.257340 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.256999 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-credential-socket\") pod \"274b53fe-9fb2-4a7a-a3a8-681930651af9\" (UID: \"274b53fe-9fb2-4a7a-a3a8-681930651af9\") " Jul 29 13:03:56.257395 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.257363 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274b53fe-9fb2-4a7a-a3a8-681930651af9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:03:56.257395 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.257374 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:03:56.257533 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.257425 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:03:56.257608 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.257581 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:03:56.258233 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.258100 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-data" (OuterVolumeSpecName: "istio-data") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:03:56.266488 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.259505 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-kube-api-access-9zmjs" (OuterVolumeSpecName: "kube-api-access-9zmjs") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "kube-api-access-9zmjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:03:56.267152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.267121 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:03:56.267275 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.267209 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-token" (OuterVolumeSpecName: "istio-token") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:03:56.269202 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.269175 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "274b53fe-9fb2-4a7a-a3a8-681930651af9" (UID: "274b53fe-9fb2-4a7a-a3a8-681930651af9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:03:56.358090 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358056 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358090 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358085 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358090 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358095 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358104 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358113 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zmjs\" (UniqueName: \"kubernetes.io/projected/274b53fe-9fb2-4a7a-a3a8-681930651af9-kube-api-access-9zmjs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358122 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358131 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358139 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274b53fe-9fb2-4a7a-a3a8-681930651af9-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.358314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.358146 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274b53fe-9fb2-4a7a-a3a8-681930651af9-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:03:56.569386 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.569296 2563 generic.go:356] "Generic (PLEG): container finished" podID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerID="506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a" exitCode=0 Jul 29 13:03:56.569386 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.569343 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" event={"ID":"274b53fe-9fb2-4a7a-a3a8-681930651af9","Type":"ContainerDied","Data":"506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a"} Jul 29 13:03:56.569386 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.569359 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" Jul 29 13:03:56.569656 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.569420 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh" event={"ID":"274b53fe-9fb2-4a7a-a3a8-681930651af9","Type":"ContainerDied","Data":"46ed0b8a25c0531e7ce46b2823b820b7925bea470e45a83c8037da40b8f87974"} Jul 29 13:03:56.569656 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.569445 2563 scope.go:117] "RemoveContainer" containerID="506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a" Jul 29 13:03:56.579191 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.579167 2563 scope.go:117] "RemoveContainer" containerID="6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd" Jul 29 13:03:56.588245 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.588228 2563 scope.go:117] "RemoveContainer" containerID="506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a" Jul 29 13:03:56.588515 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:03:56.588495 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a\": container with ID starting with 506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a not found: ID does not exist" containerID="506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a" Jul 29 13:03:56.588591 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.588521 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a"} err="failed to get container status \"506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a\": rpc error: code = NotFound desc = could not find container \"506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a\": container with ID starting with 506bef7df2f6e7b3ba64d360083c1cd4df0c543a9c2b2ecd922e0f737c37ad0a not found: ID does not exist" Jul 29 13:03:56.588591 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.588538 2563 scope.go:117] "RemoveContainer" containerID="6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd" Jul 29 13:03:56.588792 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:03:56.588775 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd\": container with ID starting with 6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd not found: ID does not exist" containerID="6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd" Jul 29 13:03:56.588837 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.588795 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd"} err="failed to get container status \"6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd\": rpc error: code = NotFound desc = could not find container \"6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd\": container with ID starting with 6fb34426927bbae0f043a13c9e8c3e59035d7ca4e88bf80e0ab87e45a33052cd not found: ID does not exist" Jul 29 13:03:56.592059 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.592032 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh"] Jul 29 13:03:56.595684 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:56.595661 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-be3b8a53-openshift-default-66cfc46bf6-k6lvh"] Jul 29 13:03:57.289358 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:57.289319 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:57.400146 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:57.400098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:57.490014 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:57.489973 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:03:57.658166 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:57.658086 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" path="/var/lib/kubelet/pods/274b53fe-9fb2-4a7a-a3a8-681930651af9/volumes" Jul 29 13:03:57.714906 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:03:57.714868 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:04:03.396192 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:03.396150 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:04:03.497229 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:03.497188 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:04:31.924269 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:31.924231 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5"] Jul 29 13:04:31.924756 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:31.924424 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" podUID="31fa3226-21d5-4abc-a89d-c403df04efe7" containerName="manager" containerID="cri-o://e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335" gracePeriod=10 Jul 29 13:04:32.675376 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.675345 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 13:04:32.724307 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.724272 2563 generic.go:356] "Generic (PLEG): container finished" podID="31fa3226-21d5-4abc-a89d-c403df04efe7" containerID="e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335" exitCode=0 Jul 29 13:04:32.724490 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.724317 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" Jul 29 13:04:32.724490 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.724360 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" event={"ID":"31fa3226-21d5-4abc-a89d-c403df04efe7","Type":"ContainerDied","Data":"e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335"} Jul 29 13:04:32.724490 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.724424 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5" event={"ID":"31fa3226-21d5-4abc-a89d-c403df04efe7","Type":"ContainerDied","Data":"0260b5f00ca60b3b01deb6e48f012e193fa2f36c86f0704a0769ef66fa50362c"} Jul 29 13:04:32.724490 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.724479 2563 scope.go:117] "RemoveContainer" containerID="e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335" Jul 29 13:04:32.736276 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.736253 2563 scope.go:117] "RemoveContainer" containerID="e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335" Jul 29 13:04:32.736566 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:04:32.736547 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335\": container with ID starting with e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335 not found: ID does not exist" containerID="e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335" Jul 29 13:04:32.736640 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.736579 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335"} err="failed to get container status \"e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335\": rpc error: code = NotFound desc = could not find container \"e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335\": container with ID starting with e102f844e65b0689446ae88c7852400fd3da389f6da1c0e576aac97cbbeef335 not found: ID does not exist" Jul 29 13:04:32.772916 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.772888 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv2pc\" (UniqueName: \"kubernetes.io/projected/31fa3226-21d5-4abc-a89d-c403df04efe7-kube-api-access-nv2pc\") pod \"31fa3226-21d5-4abc-a89d-c403df04efe7\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " Jul 29 13:04:32.773053 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.772932 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31fa3226-21d5-4abc-a89d-c403df04efe7-extensions-socket-volume\") pod \"31fa3226-21d5-4abc-a89d-c403df04efe7\" (UID: \"31fa3226-21d5-4abc-a89d-c403df04efe7\") " Jul 29 13:04:32.773426 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.773394 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31fa3226-21d5-4abc-a89d-c403df04efe7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "31fa3226-21d5-4abc-a89d-c403df04efe7" (UID: "31fa3226-21d5-4abc-a89d-c403df04efe7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:04:32.775006 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.774984 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31fa3226-21d5-4abc-a89d-c403df04efe7-kube-api-access-nv2pc" (OuterVolumeSpecName: "kube-api-access-nv2pc") pod "31fa3226-21d5-4abc-a89d-c403df04efe7" (UID: "31fa3226-21d5-4abc-a89d-c403df04efe7"). InnerVolumeSpecName "kube-api-access-nv2pc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:04:32.874110 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.874009 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv2pc\" (UniqueName: \"kubernetes.io/projected/31fa3226-21d5-4abc-a89d-c403df04efe7-kube-api-access-nv2pc\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:04:32.874110 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:32.874046 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31fa3226-21d5-4abc-a89d-c403df04efe7-extensions-socket-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:04:33.047508 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:33.047470 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5"] Jul 29 13:04:33.051808 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:33.051775 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h8rj5"] Jul 29 13:04:33.657587 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:04:33.657548 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31fa3226-21d5-4abc-a89d-c403df04efe7" path="/var/lib/kubelet/pods/31fa3226-21d5-4abc-a89d-c403df04efe7/volumes" Jul 29 13:05:47.544141 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544097 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g"] Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544604 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544626 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544652 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" containerName="payload-processing" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544661 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" containerName="payload-processing" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544682 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544691 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544705 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31fa3226-21d5-4abc-a89d-c403df04efe7" containerName="manager" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544714 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="31fa3226-21d5-4abc-a89d-c403df04efe7" containerName="manager" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544725 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" containerName="payload-pre-processing" Jul 29 13:05:47.544782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544735 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" containerName="payload-pre-processing" Jul 29 13:05:47.545274 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544813 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c698a358-36be-4f4f-995f-2b679bb733b3" containerName="payload-pre-processing" Jul 29 13:05:47.545274 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544827 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" Jul 29 13:05:47.545274 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544838 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="31fa3226-21d5-4abc-a89d-c403df04efe7" containerName="manager" Jul 29 13:05:47.545274 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544847 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6a606b9-85d3-4c54-8cec-a3510c8c1bcc" containerName="payload-processing" Jul 29 13:05:47.545274 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.544856 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="274b53fe-9fb2-4a7a-a3a8-681930651af9" containerName="istio-proxy" Jul 29 13:05:47.547572 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.547552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.550181 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.550162 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9h9ms\"" Jul 29 13:05:47.559636 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.559613 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g"] Jul 29 13:05:47.697371 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.697341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/597113a7-135f-4935-a8f2-f6b1160c1f51-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qbg8g\" (UID: \"597113a7-135f-4935-a8f2-f6b1160c1f51\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.697371 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.697373 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w489n\" (UniqueName: \"kubernetes.io/projected/597113a7-135f-4935-a8f2-f6b1160c1f51-kube-api-access-w489n\") pod \"kuadrant-operator-controller-manager-86878c769f-qbg8g\" (UID: \"597113a7-135f-4935-a8f2-f6b1160c1f51\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.798071 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.797993 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/597113a7-135f-4935-a8f2-f6b1160c1f51-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qbg8g\" (UID: \"597113a7-135f-4935-a8f2-f6b1160c1f51\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.798071 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.798025 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w489n\" (UniqueName: \"kubernetes.io/projected/597113a7-135f-4935-a8f2-f6b1160c1f51-kube-api-access-w489n\") pod \"kuadrant-operator-controller-manager-86878c769f-qbg8g\" (UID: \"597113a7-135f-4935-a8f2-f6b1160c1f51\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.798450 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.798429 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/597113a7-135f-4935-a8f2-f6b1160c1f51-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qbg8g\" (UID: \"597113a7-135f-4935-a8f2-f6b1160c1f51\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.806673 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.806652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w489n\" (UniqueName: \"kubernetes.io/projected/597113a7-135f-4935-a8f2-f6b1160c1f51-kube-api-access-w489n\") pod \"kuadrant-operator-controller-manager-86878c769f-qbg8g\" (UID: \"597113a7-135f-4935-a8f2-f6b1160c1f51\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.857606 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.857585 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:47.993316 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:47.993279 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g"] Jul 29 13:05:47.996998 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:05:47.996965 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597113a7_135f_4935_a8f2_f6b1160c1f51.slice/crio-4f76411731dbd769ef7e2b91d5332611aa86b17ab111e4edad23c22bd8e5b6bb WatchSource:0}: Error finding container 4f76411731dbd769ef7e2b91d5332611aa86b17ab111e4edad23c22bd8e5b6bb: Status 404 returned error can't find the container with id 4f76411731dbd769ef7e2b91d5332611aa86b17ab111e4edad23c22bd8e5b6bb Jul 29 13:05:48.046935 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:48.046913 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" event={"ID":"597113a7-135f-4935-a8f2-f6b1160c1f51","Type":"ContainerStarted","Data":"4f76411731dbd769ef7e2b91d5332611aa86b17ab111e4edad23c22bd8e5b6bb"} Jul 29 13:05:49.052262 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:49.052223 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" event={"ID":"597113a7-135f-4935-a8f2-f6b1160c1f51","Type":"ContainerStarted","Data":"5b7bda3a891387def63097007798d2b792f4330c6a574ee58297c4d89f588087"} Jul 29 13:05:49.052660 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:49.052284 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:05:49.075817 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:05:49.075769 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" podStartSLOduration=2.075753179 podStartE2EDuration="2.075753179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:05:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:05:49.073317804 +0000 UTC m=+2421.976514481" watchObservedRunningTime="2026-07-29 13:05:49.075753179 +0000 UTC m=+2421.978949916" Jul 29 13:06:00.059311 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:00.059282 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qbg8g" Jul 29 13:06:06.607276 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.607237 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:06.859389 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.859305 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz"] Jul 29 13:06:06.863075 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.863050 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.865422 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.865399 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-42b01b5c-openshift-default-dockercfg-nrdtt\"" Jul 29 13:06:06.873266 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.873202 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz"] Jul 29 13:06:06.951199 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-certs\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951377 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-podinfo\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951377 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-token\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951377 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951331 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e961052-6d3c-4892-b833-5ed3f55d4669-istiod-ca-cert\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951529 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951385 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55tqs\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-kube-api-access-55tqs\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951529 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-credential-socket\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951529 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951516 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-data\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951644 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-envoy\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:06.951698 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:06.951673 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-socket\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052342 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55tqs\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-kube-api-access-55tqs\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052520 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-credential-socket\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052520 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-data\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052520 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052434 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-envoy\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052521 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-socket\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-certs\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-podinfo\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052612 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-token\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e961052-6d3c-4892-b833-5ed3f55d4669-istiod-ca-cert\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052850 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052786 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-credential-socket\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.052903 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.052887 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-data\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.053077 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.053055 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-socket\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.053152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.053104 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-certs\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.053336 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.053317 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e961052-6d3c-4892-b833-5ed3f55d4669-istiod-ca-cert\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.054883 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.054860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-envoy\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.055007 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.054985 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-podinfo\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.060677 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.060659 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-token\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.060840 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.060822 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55tqs\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-kube-api-access-55tqs\") pod \"e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.178351 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.178257 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:07.315405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.315374 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz"] Jul 29 13:06:07.317568 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:06:07.317537 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e961052_6d3c_4892_b833_5ed3f55d4669.slice/crio-499c33d03a3d7c83eac947a07edecec3aef8fae476bf84aa891396105a6553cd WatchSource:0}: Error finding container 499c33d03a3d7c83eac947a07edecec3aef8fae476bf84aa891396105a6553cd: Status 404 returned error can't find the container with id 499c33d03a3d7c83eac947a07edecec3aef8fae476bf84aa891396105a6553cd Jul 29 13:06:07.320045 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.320016 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:06:07.320137 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.320097 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:06:07.320200 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:07.320144 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:06:08.141813 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:08.141779 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" event={"ID":"5e961052-6d3c-4892-b833-5ed3f55d4669","Type":"ContainerStarted","Data":"49cde6049e88907e11332c067c90859946330ddcfd40a236307af75da8084410"} Jul 29 13:06:08.142174 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:08.141834 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" event={"ID":"5e961052-6d3c-4892-b833-5ed3f55d4669","Type":"ContainerStarted","Data":"499c33d03a3d7c83eac947a07edecec3aef8fae476bf84aa891396105a6553cd"} Jul 29 13:06:08.168852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:08.168798 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" podStartSLOduration=2.168777851 podStartE2EDuration="2.168777851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:06:08.165413789 +0000 UTC m=+2441.068610467" watchObservedRunningTime="2026-07-29 13:06:08.168777851 +0000 UTC m=+2441.071974529" Jul 29 13:06:08.179275 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:08.179249 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:08.183963 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:08.183944 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:08.543234 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:08.543202 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:09.146673 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:09.146646 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:09.147638 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:09.147620 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:10.140681 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.140644 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:10.247275 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.247233 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:10.338296 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.338257 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:10.444163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.444077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:10.768948 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.768911 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:10.864575 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.864538 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:10.909799 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.909770 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t"] Jul 29 13:06:10.913206 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.913190 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:10.915431 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.915405 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-42b01b5c\"" Jul 29 13:06:10.915548 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.915518 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-42b01b5c-dockercfg-4x4td\"" Jul 29 13:06:10.923019 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.922994 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t"] Jul 29 13:06:10.946755 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.946731 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx"] Jul 29 13:06:10.949909 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.949893 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:10.957851 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.957827 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx"] Jul 29 13:06:10.984808 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.984772 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ca9d54-0ccb-4438-b728-2fd7d230b841-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:10.984985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.984838 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwd7d\" (UniqueName: \"kubernetes.io/projected/69ca9d54-0ccb-4438-b728-2fd7d230b841-kube-api-access-lwd7d\") pod \"payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:10.984985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.984924 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszqd\" (UniqueName: \"kubernetes.io/projected/a3943318-10b3-443e-99e8-ce76595e23d0-kube-api-access-tszqd\") pod \"payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:10.984985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:10.984949 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3943318-10b3-443e-99e8-ce76595e23d0-plugins-config-volume\") pod \"payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:11.086168 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.086082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszqd\" (UniqueName: \"kubernetes.io/projected/a3943318-10b3-443e-99e8-ce76595e23d0-kube-api-access-tszqd\") pod \"payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:11.086168 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.086125 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3943318-10b3-443e-99e8-ce76595e23d0-plugins-config-volume\") pod \"payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:11.086347 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.086175 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ca9d54-0ccb-4438-b728-2fd7d230b841-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:11.086347 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.086223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwd7d\" (UniqueName: \"kubernetes.io/projected/69ca9d54-0ccb-4438-b728-2fd7d230b841-kube-api-access-lwd7d\") pod \"payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:11.086900 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.086879 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ca9d54-0ccb-4438-b728-2fd7d230b841-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:11.086974 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.086926 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3943318-10b3-443e-99e8-ce76595e23d0-plugins-config-volume\") pod \"payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:11.095047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.094990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwd7d\" (UniqueName: \"kubernetes.io/projected/69ca9d54-0ccb-4438-b728-2fd7d230b841-kube-api-access-lwd7d\") pod \"payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:11.095447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.095422 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszqd\" (UniqueName: \"kubernetes.io/projected/a3943318-10b3-443e-99e8-ce76595e23d0-kube-api-access-tszqd\") pod \"payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:11.223936 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.223907 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:11.260639 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.260552 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:11.391283 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.390740 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t"] Jul 29 13:06:11.450947 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:11.450916 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx"] Jul 29 13:06:11.451903 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:06:11.451879 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ca9d54_0ccb_4438_b728_2fd7d230b841.slice/crio-ce58bc27232a340ab53ae465e321cea436e2a36256ad99d1b38bcf7779dc28b7 WatchSource:0}: Error finding container ce58bc27232a340ab53ae465e321cea436e2a36256ad99d1b38bcf7779dc28b7: Status 404 returned error can't find the container with id ce58bc27232a340ab53ae465e321cea436e2a36256ad99d1b38bcf7779dc28b7 Jul 29 13:06:12.160273 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.160237 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" event={"ID":"69ca9d54-0ccb-4438-b728-2fd7d230b841","Type":"ContainerStarted","Data":"8b13d96d481c7792725dae361816e4c83de0cf6def5109fe0be235f58ebb13de"} Jul 29 13:06:12.160603 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.160295 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" event={"ID":"69ca9d54-0ccb-4438-b728-2fd7d230b841","Type":"ContainerStarted","Data":"ce58bc27232a340ab53ae465e321cea436e2a36256ad99d1b38bcf7779dc28b7"} Jul 29 13:06:12.160603 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.160323 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:12.161852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.161827 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" event={"ID":"a3943318-10b3-443e-99e8-ce76595e23d0","Type":"ContainerStarted","Data":"b386262fdb86b40870b751df84a9e3125f61eda2c6eee89c099e43f076cd068c"} Jul 29 13:06:12.161965 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.161868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" event={"ID":"a3943318-10b3-443e-99e8-ce76595e23d0","Type":"ContainerStarted","Data":"85b563b25153ef55da7a55ae360925fb68d6be5c8c8e77ca3b519780e716c6a0"} Jul 29 13:06:12.161965 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.161907 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:12.179086 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.179040 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" podStartSLOduration=2.179025648 podStartE2EDuration="2.179025648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:06:12.176869827 +0000 UTC m=+2445.080066508" watchObservedRunningTime="2026-07-29 13:06:12.179025648 +0000 UTC m=+2445.082222325" Jul 29 13:06:12.199075 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:12.199033 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" podStartSLOduration=2.199018795 podStartE2EDuration="2.199018795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:06:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:06:12.197605002 +0000 UTC m=+2445.100801680" watchObservedRunningTime="2026-07-29 13:06:12.199018795 +0000 UTC m=+2445.102215498" Jul 29 13:06:23.169863 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:23.169819 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:06:23.170521 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:23.169916 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:06:25.188621 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:25.188574 2563 patch_prober.go:28] interesting pod/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 13:06:25.189107 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:25.188638 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 13:06:26.227959 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:26.227929 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerID="49cde6049e88907e11332c067c90859946330ddcfd40a236307af75da8084410" exitCode=0 Jul 29 13:06:26.228343 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:26.227952 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" event={"ID":"5e961052-6d3c-4892-b833-5ed3f55d4669","Type":"ContainerDied","Data":"49cde6049e88907e11332c067c90859946330ddcfd40a236307af75da8084410"} Jul 29 13:06:26.228343 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:26.228320 2563 scope.go:117] "RemoveContainer" containerID="49cde6049e88907e11332c067c90859946330ddcfd40a236307af75da8084410" Jul 29 13:06:26.229289 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:26.229265 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:06:26.229354 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:26.229329 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:06:26.229389 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:26.229361 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:06:27.233975 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:27.233941 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" event={"ID":"5e961052-6d3c-4892-b833-5ed3f55d4669","Type":"ContainerStarted","Data":"2bd5840c3f193fa94de80338a390022c32f9bddd2cd85289b3f585175be4e724"} Jul 29 13:06:28.179008 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:28.178972 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:28.179403 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:28.179375 2563 patch_prober.go:28] interesting pod/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 29 13:06:28.179531 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:28.179438 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 29 13:06:29.179552 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:29.179515 2563 patch_prober.go:28] interesting pod/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 29 13:06:29.180141 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:29.180107 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 29 13:06:30.178684 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:30.178643 2563 patch_prober.go:28] interesting pod/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 29 13:06:30.178888 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:30.178723 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 29 13:06:31.182639 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:31.182611 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:31.183038 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:31.182843 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:31.183508 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:31.183491 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:06:35.186805 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:35.186767 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:40.973694 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:40.973655 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:06:56.451380 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:06:56.451342 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:23.196580 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:23.196489 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:25.191843 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.191803 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:25.221065 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.221032 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t"] Jul 29 13:07:25.221304 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.221280 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" podUID="a3943318-10b3-443e-99e8-ce76595e23d0" containerName="payload-processing" containerID="cri-o://b386262fdb86b40870b751df84a9e3125f61eda2c6eee89c099e43f076cd068c" gracePeriod=30 Jul 29 13:07:25.230398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.229849 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx"] Jul 29 13:07:25.230398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.230065 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" podUID="69ca9d54-0ccb-4438-b728-2fd7d230b841" containerName="payload-pre-processing" containerID="cri-o://8b13d96d481c7792725dae361816e4c83de0cf6def5109fe0be235f58ebb13de" gracePeriod=30 Jul 29 13:07:25.291318 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.291280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:25.367963 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.367912 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:25.425342 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.425294 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:25.497894 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.497869 2563 generic.go:356] "Generic (PLEG): container finished" podID="69ca9d54-0ccb-4438-b728-2fd7d230b841" containerID="8b13d96d481c7792725dae361816e4c83de0cf6def5109fe0be235f58ebb13de" exitCode=0 Jul 29 13:07:25.498035 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.497938 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" event={"ID":"69ca9d54-0ccb-4438-b728-2fd7d230b841","Type":"ContainerDied","Data":"8b13d96d481c7792725dae361816e4c83de0cf6def5109fe0be235f58ebb13de"} Jul 29 13:07:25.499309 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.499280 2563 generic.go:356] "Generic (PLEG): container finished" podID="a3943318-10b3-443e-99e8-ce76595e23d0" containerID="b386262fdb86b40870b751df84a9e3125f61eda2c6eee89c099e43f076cd068c" exitCode=0 Jul 29 13:07:25.499434 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.499358 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" event={"ID":"a3943318-10b3-443e-99e8-ce76595e23d0","Type":"ContainerDied","Data":"b386262fdb86b40870b751df84a9e3125f61eda2c6eee89c099e43f076cd068c"} Jul 29 13:07:25.499434 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.499427 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" event={"ID":"a3943318-10b3-443e-99e8-ce76595e23d0","Type":"ContainerDied","Data":"85b563b25153ef55da7a55ae360925fb68d6be5c8c8e77ca3b519780e716c6a0"} Jul 29 13:07:25.499646 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.499622 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85b563b25153ef55da7a55ae360925fb68d6be5c8c8e77ca3b519780e716c6a0" Jul 29 13:07:25.502289 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.502271 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:07:25.514592 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.514575 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:07:25.661534 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.661507 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3943318-10b3-443e-99e8-ce76595e23d0-plugins-config-volume\") pod \"a3943318-10b3-443e-99e8-ce76595e23d0\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " Jul 29 13:07:25.661534 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.661540 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tszqd\" (UniqueName: \"kubernetes.io/projected/a3943318-10b3-443e-99e8-ce76595e23d0-kube-api-access-tszqd\") pod \"a3943318-10b3-443e-99e8-ce76595e23d0\" (UID: \"a3943318-10b3-443e-99e8-ce76595e23d0\") " Jul 29 13:07:25.661784 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.661591 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwd7d\" (UniqueName: \"kubernetes.io/projected/69ca9d54-0ccb-4438-b728-2fd7d230b841-kube-api-access-lwd7d\") pod \"69ca9d54-0ccb-4438-b728-2fd7d230b841\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " Jul 29 13:07:25.661784 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.661616 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ca9d54-0ccb-4438-b728-2fd7d230b841-plugins-config-volume\") pod \"69ca9d54-0ccb-4438-b728-2fd7d230b841\" (UID: \"69ca9d54-0ccb-4438-b728-2fd7d230b841\") " Jul 29 13:07:25.662006 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.661976 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3943318-10b3-443e-99e8-ce76595e23d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3943318-10b3-443e-99e8-ce76595e23d0" (UID: "a3943318-10b3-443e-99e8-ce76595e23d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:07:25.662105 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.662077 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ca9d54-0ccb-4438-b728-2fd7d230b841-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69ca9d54-0ccb-4438-b728-2fd7d230b841" (UID: "69ca9d54-0ccb-4438-b728-2fd7d230b841"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:07:25.663696 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.663670 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ca9d54-0ccb-4438-b728-2fd7d230b841-kube-api-access-lwd7d" (OuterVolumeSpecName: "kube-api-access-lwd7d") pod "69ca9d54-0ccb-4438-b728-2fd7d230b841" (UID: "69ca9d54-0ccb-4438-b728-2fd7d230b841"). InnerVolumeSpecName "kube-api-access-lwd7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:07:25.664045 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.664030 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3943318-10b3-443e-99e8-ce76595e23d0-kube-api-access-tszqd" (OuterVolumeSpecName: "kube-api-access-tszqd") pod "a3943318-10b3-443e-99e8-ce76595e23d0" (UID: "a3943318-10b3-443e-99e8-ce76595e23d0"). InnerVolumeSpecName "kube-api-access-tszqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:07:25.763258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.763219 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3943318-10b3-443e-99e8-ce76595e23d0-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:07:25.763258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.763251 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tszqd\" (UniqueName: \"kubernetes.io/projected/a3943318-10b3-443e-99e8-ce76595e23d0-kube-api-access-tszqd\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:07:25.763258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.763264 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwd7d\" (UniqueName: \"kubernetes.io/projected/69ca9d54-0ccb-4438-b728-2fd7d230b841-kube-api-access-lwd7d\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:07:25.763516 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:25.763277 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ca9d54-0ccb-4438-b728-2fd7d230b841-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:07:26.505420 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.505333 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" Jul 29 13:07:26.505420 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.505380 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx" event={"ID":"69ca9d54-0ccb-4438-b728-2fd7d230b841","Type":"ContainerDied","Data":"ce58bc27232a340ab53ae465e321cea436e2a36256ad99d1b38bcf7779dc28b7"} Jul 29 13:07:26.505905 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.505438 2563 scope.go:117] "RemoveContainer" containerID="8b13d96d481c7792725dae361816e4c83de0cf6def5109fe0be235f58ebb13de" Jul 29 13:07:26.505905 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.505577 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t" Jul 29 13:07:26.529488 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.529444 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t"] Jul 29 13:07:26.531794 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.531772 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-42b01b5c-869c979694-8tk8t"] Jul 29 13:07:26.542150 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.542121 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx"] Jul 29 13:07:26.550743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:26.550707 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-42b01b5c-57fd54f84-89dfx"] Jul 29 13:07:27.658568 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:27.658536 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ca9d54-0ccb-4438-b728-2fd7d230b841" path="/var/lib/kubelet/pods/69ca9d54-0ccb-4438-b728-2fd7d230b841/volumes" Jul 29 13:07:27.658968 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:27.658952 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3943318-10b3-443e-99e8-ce76595e23d0" path="/var/lib/kubelet/pods/a3943318-10b3-443e-99e8-ce76595e23d0/volumes" Jul 29 13:07:28.687775 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:28.687742 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:07:33.786873 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:07:33.786830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:10.614951 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:10.614909 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:10.798374 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:10.798336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:16.545195 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:16.545163 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:22.394962 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:22.394921 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:22.495569 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:22.495523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:22.650544 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:22.650445 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz"] Jul 29 13:08:22.650719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:22.650698 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" containerID="cri-o://2bd5840c3f193fa94de80338a390022c32f9bddd2cd85289b3f585175be4e724" gracePeriod=30 Jul 29 13:08:22.677167 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:22.677139 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:22.999879 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:22.999842 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:27.786149 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:27.786112 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerID="2bd5840c3f193fa94de80338a390022c32f9bddd2cd85289b3f585175be4e724" exitCode=0 Jul 29 13:08:27.786539 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:27.786177 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" event={"ID":"5e961052-6d3c-4892-b833-5ed3f55d4669","Type":"ContainerDied","Data":"2bd5840c3f193fa94de80338a390022c32f9bddd2cd85289b3f585175be4e724"} Jul 29 13:08:27.786539 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:27.786241 2563 scope.go:117] "RemoveContainer" containerID="49cde6049e88907e11332c067c90859946330ddcfd40a236307af75da8084410" Jul 29 13:08:27.903420 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:27.903392 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:08:28.002157 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002115 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-certs\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002157 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002161 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-envoy\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002419 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002184 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-podinfo\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002419 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002206 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-socket\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002419 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002238 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55tqs\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-kube-api-access-55tqs\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002419 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002270 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-credential-socket\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002419 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002298 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-data\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002419 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002420 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-token\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002448 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e961052-6d3c-4892-b833-5ed3f55d4669-istiod-ca-cert\") pod \"5e961052-6d3c-4892-b833-5ed3f55d4669\" (UID: \"5e961052-6d3c-4892-b833-5ed3f55d4669\") " Jul 29 13:08:28.002780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002517 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:08:28.002780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002627 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:08:28.002780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002634 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:08:28.002780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002762 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.002780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002779 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.003036 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002795 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.003036 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.002935 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e961052-6d3c-4892-b833-5ed3f55d4669-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:08:28.003415 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.003392 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:08:28.004954 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.004933 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:08:28.005058 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.004946 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:08:28.005058 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.005023 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:08:28.005145 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.005046 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-kube-api-access-55tqs" (OuterVolumeSpecName: "kube-api-access-55tqs") pod "5e961052-6d3c-4892-b833-5ed3f55d4669" (UID: "5e961052-6d3c-4892-b833-5ed3f55d4669"). InnerVolumeSpecName "kube-api-access-55tqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:08:28.103926 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.103884 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.103926 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.103917 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e961052-6d3c-4892-b833-5ed3f55d4669-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.103926 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.103926 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.103926 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.103934 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.104242 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.103943 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55tqs\" (UniqueName: \"kubernetes.io/projected/5e961052-6d3c-4892-b833-5ed3f55d4669-kube-api-access-55tqs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.104242 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.103953 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e961052-6d3c-4892-b833-5ed3f55d4669-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:08:28.791557 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.791527 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" Jul 29 13:08:28.791991 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.791565 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz" event={"ID":"5e961052-6d3c-4892-b833-5ed3f55d4669","Type":"ContainerDied","Data":"499c33d03a3d7c83eac947a07edecec3aef8fae476bf84aa891396105a6553cd"} Jul 29 13:08:28.791991 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.791625 2563 scope.go:117] "RemoveContainer" containerID="2bd5840c3f193fa94de80338a390022c32f9bddd2cd85289b3f585175be4e724" Jul 29 13:08:28.813056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.813023 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz"] Jul 29 13:08:28.816818 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:28.816776 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-42b01b5c-openshift-default-5645bf6cd4-xd2tz"] Jul 29 13:08:29.658224 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:29.658190 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" path="/var/lib/kubelet/pods/5e961052-6d3c-4892-b833-5ed3f55d4669/volumes" Jul 29 13:08:34.317248 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:34.317208 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:34.418133 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:34.418094 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:34.517733 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:34.517701 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:45.546883 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:45.546841 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:45.645231 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:45.645188 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:45.745341 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:45.745295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:47.228876 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:47.228843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:53.003021 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:53.002946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:53.052392 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:53.052356 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:53.699223 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:53.699192 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:59.377636 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:59.377596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:59.564310 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:59.564256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:08:59.665106 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:08:59.665009 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:10.128818 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:10.128775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:20.737553 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:20.737512 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:29.559214 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:29.559172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:40.581349 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:40.581310 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:41.774477 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:41.774434 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:47.959133 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:47.959098 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:48.081004 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.080963 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:48.368210 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.368122 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:48.664604 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664529 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl"] Jul 29 13:09:48.664859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664847 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" Jul 29 13:09:48.664902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664861 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" Jul 29 13:09:48.664902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664878 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3943318-10b3-443e-99e8-ce76595e23d0" containerName="payload-processing" Jul 29 13:09:48.664902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664884 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3943318-10b3-443e-99e8-ce76595e23d0" containerName="payload-processing" Jul 29 13:09:48.664902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664893 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ca9d54-0ccb-4438-b728-2fd7d230b841" containerName="payload-pre-processing" Jul 29 13:09:48.664902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664899 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ca9d54-0ccb-4438-b728-2fd7d230b841" containerName="payload-pre-processing" Jul 29 13:09:48.665092 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664952 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" Jul 29 13:09:48.665092 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664962 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ca9d54-0ccb-4438-b728-2fd7d230b841" containerName="payload-pre-processing" Jul 29 13:09:48.665092 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.664968 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3943318-10b3-443e-99e8-ce76595e23d0" containerName="payload-processing" Jul 29 13:09:48.667948 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.667926 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.670206 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.670186 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a7e361-openshift-default-dockercfg-646kx\"" Jul 29 13:09:48.681407 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.681382 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl"] Jul 29 13:09:48.685841 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.685809 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:48.773445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773410 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-envoy\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773635 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773452 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-credential-socket\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773635 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-socket\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773635 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-token\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773635 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47e1fd3-e20b-456a-84b1-65524f29abfc-istiod-ca-cert\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773809 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqmn\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-kube-api-access-mvqmn\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773809 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-certs\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773883 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-data\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.773883 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.773831 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-podinfo\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.874451 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-data\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.874620 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874487 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-podinfo\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.874620 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874535 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-envoy\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.874620 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874565 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-credential-socket\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-data\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874878 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-socket\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874951 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-token\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.874984 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-credential-socket\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.875003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47e1fd3-e20b-456a-84b1-65524f29abfc-istiod-ca-cert\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.875031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqmn\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-kube-api-access-mvqmn\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.875080 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-certs\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.875170 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-socket\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.875445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.875400 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-certs\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.876013 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.875669 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47e1fd3-e20b-456a-84b1-65524f29abfc-istiod-ca-cert\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.877045 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.877018 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-envoy\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.877651 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.877631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-podinfo\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.882644 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.882620 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-token\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.882988 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.882970 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqmn\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-kube-api-access-mvqmn\") pod \"e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:48.978290 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:48.978268 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:49.113604 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:49.113573 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl"] Jul 29 13:09:49.116251 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:09:49.116219 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47e1fd3_e20b_456a_84b1_65524f29abfc.slice/crio-1af744dd2b026fa269d5995e9363212d0f224b9006c7f652df7152a34e810523 WatchSource:0}: Error finding container 1af744dd2b026fa269d5995e9363212d0f224b9006c7f652df7152a34e810523: Status 404 returned error can't find the container with id 1af744dd2b026fa269d5995e9363212d0f224b9006c7f652df7152a34e810523 Jul 29 13:09:49.121737 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:49.121705 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:09:49.121835 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:49.121768 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:09:49.121835 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:49.121799 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:09:49.149365 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:49.149344 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" event={"ID":"d47e1fd3-e20b-456a-84b1-65524f29abfc","Type":"ContainerStarted","Data":"1af744dd2b026fa269d5995e9363212d0f224b9006c7f652df7152a34e810523"} Jul 29 13:09:49.234012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:49.233929 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:50.161886 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:50.161846 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" event={"ID":"d47e1fd3-e20b-456a-84b1-65524f29abfc","Type":"ContainerStarted","Data":"1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59"} Jul 29 13:09:50.184024 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:50.183965 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podStartSLOduration=2.183945654 podStartE2EDuration="2.183945654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:09:50.18328589 +0000 UTC m=+2663.086482567" watchObservedRunningTime="2026-07-29 13:09:50.183945654 +0000 UTC m=+2663.087142342" Jul 29 13:09:50.979070 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:50.979038 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:50.983574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:50.983548 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:51.166633 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:51.166601 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:51.167451 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:51.167432 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:09:52.154040 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.153984 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:52.232318 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.232286 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:52.325940 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.325903 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:52.519858 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.519825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:52.715298 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.715262 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:52.815609 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.815523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:09:52.827183 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.827150 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj"] Jul 29 13:09:52.827509 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.827496 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" Jul 29 13:09:52.827566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.827510 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" Jul 29 13:09:52.827603 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.827579 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e961052-6d3c-4892-b833-5ed3f55d4669" containerName="istio-proxy" Jul 29 13:09:52.830404 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.830387 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:52.832819 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.832799 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-a7e361\"" Jul 29 13:09:52.832905 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.832808 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-a7e361-dockercfg-4qllt\"" Jul 29 13:09:52.840423 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.840402 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj"] Jul 29 13:09:52.843023 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.843001 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm"] Jul 29 13:09:52.850775 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.850756 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:52.857121 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.857084 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm"] Jul 29 13:09:52.910180 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.910153 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nlc\" (UniqueName: \"kubernetes.io/projected/b2d53ce5-eba9-4843-856d-563e280de8b4-kube-api-access-g9nlc\") pod \"payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:52.910278 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:52.910239 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d53ce5-eba9-4843-856d-563e280de8b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:53.010848 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.010813 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d53ce5-eba9-4843-856d-563e280de8b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:53.011044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.010869 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5jtx\" (UniqueName: \"kubernetes.io/projected/bdfdd954-534e-4b5b-bf65-ef138cedebb1-kube-api-access-q5jtx\") pod \"payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.011044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.010910 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdfdd954-534e-4b5b-bf65-ef138cedebb1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.011044 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.010970 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nlc\" (UniqueName: \"kubernetes.io/projected/b2d53ce5-eba9-4843-856d-563e280de8b4-kube-api-access-g9nlc\") pod \"payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:53.011572 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.011552 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d53ce5-eba9-4843-856d-563e280de8b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:53.020383 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.020350 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nlc\" (UniqueName: \"kubernetes.io/projected/b2d53ce5-eba9-4843-856d-563e280de8b4-kube-api-access-g9nlc\") pod \"payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:53.112362 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.112284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5jtx\" (UniqueName: \"kubernetes.io/projected/bdfdd954-534e-4b5b-bf65-ef138cedebb1-kube-api-access-q5jtx\") pod \"payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.112362 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.112325 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdfdd954-534e-4b5b-bf65-ef138cedebb1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.113007 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.112984 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdfdd954-534e-4b5b-bf65-ef138cedebb1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.121771 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.121750 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5jtx\" (UniqueName: \"kubernetes.io/projected/bdfdd954-534e-4b5b-bf65-ef138cedebb1-kube-api-access-q5jtx\") pod \"payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.141729 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.141705 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:53.162053 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.162032 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:53.287431 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.287400 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj"] Jul 29 13:09:53.289741 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:09:53.289710 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d53ce5_eba9_4843_856d_563e280de8b4.slice/crio-6cb82b4feff650bbdc960697253e88062d82bf15f75df8f7e2ef9f5085bbeac5 WatchSource:0}: Error finding container 6cb82b4feff650bbdc960697253e88062d82bf15f75df8f7e2ef9f5085bbeac5: Status 404 returned error can't find the container with id 6cb82b4feff650bbdc960697253e88062d82bf15f75df8f7e2ef9f5085bbeac5 Jul 29 13:09:53.314946 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:53.314924 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm"] Jul 29 13:09:53.328482 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:09:53.328446 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdfdd954_534e_4b5b_bf65_ef138cedebb1.slice/crio-06a7ebbb413cb5fc7f8eebb05f4d009af1238c322ec1d71524d1a97f60366226 WatchSource:0}: Error finding container 06a7ebbb413cb5fc7f8eebb05f4d009af1238c322ec1d71524d1a97f60366226: Status 404 returned error can't find the container with id 06a7ebbb413cb5fc7f8eebb05f4d009af1238c322ec1d71524d1a97f60366226 Jul 29 13:09:54.185635 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.185604 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" event={"ID":"bdfdd954-534e-4b5b-bf65-ef138cedebb1","Type":"ContainerStarted","Data":"c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168"} Jul 29 13:09:54.185826 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.185658 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" event={"ID":"bdfdd954-534e-4b5b-bf65-ef138cedebb1","Type":"ContainerStarted","Data":"06a7ebbb413cb5fc7f8eebb05f4d009af1238c322ec1d71524d1a97f60366226"} Jul 29 13:09:54.185826 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.185718 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:09:54.187305 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.187282 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" event={"ID":"b2d53ce5-eba9-4843-856d-563e280de8b4","Type":"ContainerStarted","Data":"a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0"} Jul 29 13:09:54.187405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.187324 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:09:54.187405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.187341 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" event={"ID":"b2d53ce5-eba9-4843-856d-563e280de8b4","Type":"ContainerStarted","Data":"6cb82b4feff650bbdc960697253e88062d82bf15f75df8f7e2ef9f5085bbeac5"} Jul 29 13:09:54.203356 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.203295 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" podStartSLOduration=2.203282214 podStartE2EDuration="2.203282214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:09:54.20167694 +0000 UTC m=+2667.104873622" watchObservedRunningTime="2026-07-29 13:09:54.203282214 +0000 UTC m=+2667.106478893" Jul 29 13:09:54.218944 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:09:54.218896 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" podStartSLOduration=2.218882878 podStartE2EDuration="2.218882878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:09:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:09:54.218656939 +0000 UTC m=+2667.121853617" watchObservedRunningTime="2026-07-29 13:09:54.218882878 +0000 UTC m=+2667.122079554" Jul 29 13:10:05.194669 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.194645 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:10:05.195041 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.194713 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:10:05.241202 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.241166 2563 generic.go:356] "Generic (PLEG): container finished" podID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerID="1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59" exitCode=0 Jul 29 13:10:05.241348 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.241236 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" event={"ID":"d47e1fd3-e20b-456a-84b1-65524f29abfc","Type":"ContainerDied","Data":"1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59"} Jul 29 13:10:05.241645 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.241629 2563 scope.go:117] "RemoveContainer" containerID="1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59" Jul 29 13:10:05.242657 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.242627 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:05.242735 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.242702 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:05.242778 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:05.242751 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:06.166825 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:06.166795 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:10:06.253443 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:06.253268 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" event={"ID":"d47e1fd3-e20b-456a-84b1-65524f29abfc","Type":"ContainerStarted","Data":"619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980"} Jul 29 13:10:06.980624 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:06.980580 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:10:06.980844 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:06.980712 2563 patch_prober.go:28] interesting pod/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 29 13:10:06.980844 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:06.980774 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 29 13:10:07.978866 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:07.978825 2563 patch_prober.go:28] interesting pod/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 29 13:10:07.979501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:07.979437 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 29 13:10:08.978976 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:08.978933 2563 patch_prober.go:28] interesting pod/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 29 13:10:08.979479 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:08.979011 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 29 13:10:09.980840 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:09.980806 2563 patch_prober.go:28] interesting pod/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 29 13:10:09.981388 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:09.981358 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 29 13:10:10.982933 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:10.982901 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:10:10.983336 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:10.983188 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:10:10.983838 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:10.983818 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:10:17.223335 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.223297 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:17.554631 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.554545 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:17.832407 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.832318 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt"] Jul 29 13:10:17.835034 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.835015 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.837498 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.837478 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ef0894-openshift-default-dockercfg-k52jp\"" Jul 29 13:10:17.851944 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.851919 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt"] Jul 29 13:10:17.872963 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.872303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:17.911667 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911632 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-certs\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911681 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9gd\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-kube-api-access-rc9gd\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911780 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911743 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-token\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911783 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-socket\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911813 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-envoy\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911827 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-podinfo\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911956 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911909 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-credential-socket\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.911956 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911930 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0044f27f-90a5-4207-bdc5-561f129c6a10-istiod-ca-cert\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:17.912020 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:17.911998 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-data\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.012946 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.012915 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-credential-socket\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.012946 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.012947 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0044f27f-90a5-4207-bdc5-561f129c6a10-istiod-ca-cert\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.012973 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-data\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-certs\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013029 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9gd\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-kube-api-access-rc9gd\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013059 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-token\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013077 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-socket\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-envoy\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013134 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-podinfo\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013592 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013402 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-credential-socket\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013592 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-socket\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013702 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013661 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-data\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013835 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013811 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0044f27f-90a5-4207-bdc5-561f129c6a10-istiod-ca-cert\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.013890 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.013828 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-certs\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.015409 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.015390 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-envoy\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.015574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.015557 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-podinfo\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.020405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.020386 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-token\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.020606 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.020588 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9gd\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-kube-api-access-rc9gd\") pod \"e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.144674 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.144601 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:18.289777 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.289751 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt"] Jul 29 13:10:18.291643 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:10:18.291605 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0044f27f_90a5_4207_bdc5_561f129c6a10.slice/crio-42871a0db4598f919c6e3cd3d2175afa4ebf212524ece894d4dfe11ade30da14 WatchSource:0}: Error finding container 42871a0db4598f919c6e3cd3d2175afa4ebf212524ece894d4dfe11ade30da14: Status 404 returned error can't find the container with id 42871a0db4598f919c6e3cd3d2175afa4ebf212524ece894d4dfe11ade30da14 Jul 29 13:10:18.294077 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.294046 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:18.294172 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.294123 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:18.294172 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.294165 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:18.315351 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.315326 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" event={"ID":"0044f27f-90a5-4207-bdc5-561f129c6a10","Type":"ContainerStarted","Data":"42871a0db4598f919c6e3cd3d2175afa4ebf212524ece894d4dfe11ade30da14"} Jul 29 13:10:18.415075 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:18.414982 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:19.323758 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:19.323723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" event={"ID":"0044f27f-90a5-4207-bdc5-561f129c6a10","Type":"ContainerStarted","Data":"5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8"} Jul 29 13:10:19.352391 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:19.352334 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" podStartSLOduration=2.352314479 podStartE2EDuration="2.352314479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:10:19.349069986 +0000 UTC m=+2692.252266664" watchObservedRunningTime="2026-07-29 13:10:19.352314479 +0000 UTC m=+2692.255511158" Jul 29 13:10:20.145395 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:20.145364 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:20.150377 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:20.150348 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:20.329268 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:20.329244 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:20.329986 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:20.329968 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:21.353823 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:21.353786 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:21.523332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:21.523291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:21.622409 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:21.622323 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:21.767878 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:21.767843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:22.097445 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.097411 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:22.196985 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.196950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:22.202202 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.202176 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484"] Jul 29 13:10:22.204560 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.204545 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.207493 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.207450 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ef0894\"" Jul 29 13:10:22.207607 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.207474 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ef0894-dockercfg-jkfjf\"" Jul 29 13:10:22.218101 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.218076 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484"] Jul 29 13:10:22.249325 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.249284 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a44a66d-7047-4bf7-95ec-fbf4763bf559-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.249428 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.249389 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzgvx\" (UniqueName: \"kubernetes.io/projected/0a44a66d-7047-4bf7-95ec-fbf4763bf559-kube-api-access-xzgvx\") pod \"payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.350847 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.350762 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a44a66d-7047-4bf7-95ec-fbf4763bf559-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.351035 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.350867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzgvx\" (UniqueName: \"kubernetes.io/projected/0a44a66d-7047-4bf7-95ec-fbf4763bf559-kube-api-access-xzgvx\") pod \"payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.351471 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.351438 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a44a66d-7047-4bf7-95ec-fbf4763bf559-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.359758 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.359737 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzgvx\" (UniqueName: \"kubernetes.io/projected/0a44a66d-7047-4bf7-95ec-fbf4763bf559-kube-api-access-xzgvx\") pod \"payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.515583 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.515551 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:22.651828 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:22.651803 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484"] Jul 29 13:10:22.654525 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:10:22.654495 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a44a66d_7047_4bf7_95ec_fbf4763bf559.slice/crio-685d89548fa7ad227fe8f867c3a4975cb552e2b5ecc7b7ba53274d528912e8ee WatchSource:0}: Error finding container 685d89548fa7ad227fe8f867c3a4975cb552e2b5ecc7b7ba53274d528912e8ee: Status 404 returned error can't find the container with id 685d89548fa7ad227fe8f867c3a4975cb552e2b5ecc7b7ba53274d528912e8ee Jul 29 13:10:23.345525 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:23.345451 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" event={"ID":"0a44a66d-7047-4bf7-95ec-fbf4763bf559","Type":"ContainerStarted","Data":"acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8"} Jul 29 13:10:23.345759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:23.345581 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" event={"ID":"0a44a66d-7047-4bf7-95ec-fbf4763bf559","Type":"ContainerStarted","Data":"685d89548fa7ad227fe8f867c3a4975cb552e2b5ecc7b7ba53274d528912e8ee"} Jul 29 13:10:23.345759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:23.345610 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:23.362305 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:23.362258 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" podStartSLOduration=1.36224359 podStartE2EDuration="1.36224359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:10:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:10:23.360288515 +0000 UTC m=+2696.263485205" watchObservedRunningTime="2026-07-29 13:10:23.36224359 +0000 UTC m=+2696.265440266" Jul 29 13:10:34.352415 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:34.352387 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:10:36.371668 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:36.371623 2563 patch_prober.go:28] interesting pod/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 13:10:36.372060 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:36.371698 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 13:10:37.418360 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:37.418331 2563 generic.go:356] "Generic (PLEG): container finished" podID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerID="5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8" exitCode=0 Jul 29 13:10:37.418812 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:37.418409 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" event={"ID":"0044f27f-90a5-4207-bdc5-561f129c6a10","Type":"ContainerDied","Data":"5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8"} Jul 29 13:10:37.418812 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:37.418762 2563 scope.go:117] "RemoveContainer" containerID="5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8" Jul 29 13:10:37.419702 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:37.419675 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:37.419755 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:37.419743 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:37.419790 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:37.419778 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:10:38.425055 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:38.425021 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" event={"ID":"0044f27f-90a5-4207-bdc5-561f129c6a10","Type":"ContainerStarted","Data":"d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4"} Jul 29 13:10:39.145434 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:39.145401 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:39.145651 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:39.145629 2563 patch_prober.go:28] interesting pod/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 29 13:10:39.145705 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:39.145687 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 29 13:10:40.145674 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:40.145641 2563 patch_prober.go:28] interesting pod/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 29 13:10:40.146046 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:40.145703 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 29 13:10:41.145833 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:41.145785 2563 patch_prober.go:28] interesting pod/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 29 13:10:41.146196 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:41.145861 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 29 13:10:42.148268 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:42.148232 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:42.148654 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:42.148497 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:42.149104 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:42.149086 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:10:46.384224 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:46.384170 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:10:54.383496 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:10:54.383451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:11:20.122115 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:11:20.122081 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:06.457645 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:06.457601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:07.494224 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:07.494189 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:18.152949 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.152910 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:18.181553 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.181520 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm"] Jul 29 13:12:18.181803 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.181733 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" podUID="bdfdd954-534e-4b5b-bf65-ef138cedebb1" containerName="payload-processing" containerID="cri-o://c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168" gracePeriod=30 Jul 29 13:12:18.190805 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.190764 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj"] Jul 29 13:12:18.191814 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.191784 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" podUID="b2d53ce5-eba9-4843-856d-563e280de8b4" containerName="payload-pre-processing" containerID="cri-o://a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0" gracePeriod=30 Jul 29 13:12:18.225769 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.225742 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:18.323553 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.323518 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:18.409583 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.409485 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:18.477381 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.477352 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:12:18.480148 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.480131 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:12:18.509596 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.509575 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d53ce5-eba9-4843-856d-563e280de8b4-plugins-config-volume\") pod \"b2d53ce5-eba9-4843-856d-563e280de8b4\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " Jul 29 13:12:18.509679 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.509619 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdfdd954-534e-4b5b-bf65-ef138cedebb1-plugins-config-volume\") pod \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " Jul 29 13:12:18.509720 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.509691 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5jtx\" (UniqueName: \"kubernetes.io/projected/bdfdd954-534e-4b5b-bf65-ef138cedebb1-kube-api-access-q5jtx\") pod \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\" (UID: \"bdfdd954-534e-4b5b-bf65-ef138cedebb1\") " Jul 29 13:12:18.509756 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.509722 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9nlc\" (UniqueName: \"kubernetes.io/projected/b2d53ce5-eba9-4843-856d-563e280de8b4-kube-api-access-g9nlc\") pod \"b2d53ce5-eba9-4843-856d-563e280de8b4\" (UID: \"b2d53ce5-eba9-4843-856d-563e280de8b4\") " Jul 29 13:12:18.509964 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.509944 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d53ce5-eba9-4843-856d-563e280de8b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2d53ce5-eba9-4843-856d-563e280de8b4" (UID: "b2d53ce5-eba9-4843-856d-563e280de8b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:12:18.510009 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.509989 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdfdd954-534e-4b5b-bf65-ef138cedebb1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bdfdd954-534e-4b5b-bf65-ef138cedebb1" (UID: "bdfdd954-534e-4b5b-bf65-ef138cedebb1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:12:18.510009 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.510002 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d53ce5-eba9-4843-856d-563e280de8b4-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:18.511659 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.511639 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d53ce5-eba9-4843-856d-563e280de8b4-kube-api-access-g9nlc" (OuterVolumeSpecName: "kube-api-access-g9nlc") pod "b2d53ce5-eba9-4843-856d-563e280de8b4" (UID: "b2d53ce5-eba9-4843-856d-563e280de8b4"). InnerVolumeSpecName "kube-api-access-g9nlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:18.511803 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.511782 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdfdd954-534e-4b5b-bf65-ef138cedebb1-kube-api-access-q5jtx" (OuterVolumeSpecName: "kube-api-access-q5jtx") pod "bdfdd954-534e-4b5b-bf65-ef138cedebb1" (UID: "bdfdd954-534e-4b5b-bf65-ef138cedebb1"). InnerVolumeSpecName "kube-api-access-q5jtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:18.610700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.610668 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdfdd954-534e-4b5b-bf65-ef138cedebb1-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:18.610700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.610694 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5jtx\" (UniqueName: \"kubernetes.io/projected/bdfdd954-534e-4b5b-bf65-ef138cedebb1-kube-api-access-q5jtx\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:18.610700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.610704 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9nlc\" (UniqueName: \"kubernetes.io/projected/b2d53ce5-eba9-4843-856d-563e280de8b4-kube-api-access-g9nlc\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:18.781678 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.781648 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:18.894698 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.894666 2563 generic.go:356] "Generic (PLEG): container finished" podID="bdfdd954-534e-4b5b-bf65-ef138cedebb1" containerID="c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168" exitCode=0 Jul 29 13:12:18.894855 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.894711 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" Jul 29 13:12:18.894855 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.894747 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" event={"ID":"bdfdd954-534e-4b5b-bf65-ef138cedebb1","Type":"ContainerDied","Data":"c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168"} Jul 29 13:12:18.894855 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.894801 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm" event={"ID":"bdfdd954-534e-4b5b-bf65-ef138cedebb1","Type":"ContainerDied","Data":"06a7ebbb413cb5fc7f8eebb05f4d009af1238c322ec1d71524d1a97f60366226"} Jul 29 13:12:18.894855 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.894824 2563 scope.go:117] "RemoveContainer" containerID="c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168" Jul 29 13:12:18.896058 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.896035 2563 generic.go:356] "Generic (PLEG): container finished" podID="b2d53ce5-eba9-4843-856d-563e280de8b4" containerID="a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0" exitCode=0 Jul 29 13:12:18.896164 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.896068 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" Jul 29 13:12:18.896164 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.896069 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" event={"ID":"b2d53ce5-eba9-4843-856d-563e280de8b4","Type":"ContainerDied","Data":"a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0"} Jul 29 13:12:18.896249 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.896181 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj" event={"ID":"b2d53ce5-eba9-4843-856d-563e280de8b4","Type":"ContainerDied","Data":"6cb82b4feff650bbdc960697253e88062d82bf15f75df8f7e2ef9f5085bbeac5"} Jul 29 13:12:18.904559 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.904526 2563 scope.go:117] "RemoveContainer" containerID="c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168" Jul 29 13:12:18.904832 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:18.904813 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168\": container with ID starting with c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168 not found: ID does not exist" containerID="c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168" Jul 29 13:12:18.904904 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.904838 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168"} err="failed to get container status \"c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168\": rpc error: code = NotFound desc = could not find container \"c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168\": container with ID starting with c743feaa7bd2cd34143e89e3a9a7f54af69cf612ddb668e318cc73e0eac48168 not found: ID does not exist" Jul 29 13:12:18.904904 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.904852 2563 scope.go:117] "RemoveContainer" containerID="a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0" Jul 29 13:12:18.915277 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.915261 2563 scope.go:117] "RemoveContainer" containerID="a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0" Jul 29 13:12:18.915621 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:18.915598 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0\": container with ID starting with a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0 not found: ID does not exist" containerID="a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0" Jul 29 13:12:18.915703 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.915627 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0"} err="failed to get container status \"a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0\": rpc error: code = NotFound desc = could not find container \"a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0\": container with ID starting with a33f9eb791d62884750575fd823a61dd78680060d69d669692210437af951af0 not found: ID does not exist" Jul 29 13:12:18.919513 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.919493 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm"] Jul 29 13:12:18.923611 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.923591 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-a7e361-7995ddd484-r7npm"] Jul 29 13:12:18.934181 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.934161 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj"] Jul 29 13:12:18.937990 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:18.937971 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-a7e361-6f4fd45f44-blsxj"] Jul 29 13:12:19.658725 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:19.658697 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d53ce5-eba9-4843-856d-563e280de8b4" path="/var/lib/kubelet/pods/b2d53ce5-eba9-4843-856d-563e280de8b4/volumes" Jul 29 13:12:19.659082 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:19.659024 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdfdd954-534e-4b5b-bf65-ef138cedebb1" path="/var/lib/kubelet/pods/bdfdd954-534e-4b5b-bf65-ef138cedebb1/volumes" Jul 29 13:12:24.681483 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:24.681436 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:24.781980 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:24.781948 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:24.939834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:24.939751 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl"] Jul 29 13:12:24.939987 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:24.939954 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" containerID="cri-o://619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980" gracePeriod=30 Jul 29 13:12:24.953916 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:24.953889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:25.200792 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:25.200711 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:25.986150 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:25.986119 2563 patch_prober.go:28] interesting pod/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 13:12:25.986526 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:25.986175 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 13:12:27.850549 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:27.850507 2563 scope.go:117] "RemoveContainer" containerID="b386262fdb86b40870b751df84a9e3125f61eda2c6eee89c099e43f076cd068c" Jul 29 13:12:30.187693 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.187671 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:12:30.308297 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308262 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-credential-socket\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308329 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-podinfo\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308373 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-socket\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308399 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvqmn\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-kube-api-access-mvqmn\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308590 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308514 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-certs\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308590 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308565 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-data\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308579 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:30.308700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308599 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47e1fd3-e20b-456a-84b1-65524f29abfc-istiod-ca-cert\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308626 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-envoy\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308657 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-token\") pod \"d47e1fd3-e20b-456a-84b1-65524f29abfc\" (UID: \"d47e1fd3-e20b-456a-84b1-65524f29abfc\") " Jul 29 13:12:30.308700 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308660 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:30.308935 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308778 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:30.309029 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.308995 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d47e1fd3-e20b-456a-84b1-65524f29abfc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:12:30.309029 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.309016 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.309193 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.309039 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.309193 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.309053 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.309842 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.309816 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:30.310625 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.310605 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:12:30.310713 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.310690 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-kube-api-access-mvqmn" (OuterVolumeSpecName: "kube-api-access-mvqmn") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "kube-api-access-mvqmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:30.310713 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.310702 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:30.310789 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.310703 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d47e1fd3-e20b-456a-84b1-65524f29abfc" (UID: "d47e1fd3-e20b-456a-84b1-65524f29abfc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:30.410126 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.410094 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.410126 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.410127 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d47e1fd3-e20b-456a-84b1-65524f29abfc-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.410126 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.410137 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.410291 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.410144 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.410291 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.410152 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d47e1fd3-e20b-456a-84b1-65524f29abfc-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.410291 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.410159 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvqmn\" (UniqueName: \"kubernetes.io/projected/d47e1fd3-e20b-456a-84b1-65524f29abfc-kube-api-access-mvqmn\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:30.957569 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.957537 2563 generic.go:356] "Generic (PLEG): container finished" podID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerID="619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980" exitCode=0 Jul 29 13:12:30.957719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.957579 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" Jul 29 13:12:30.957719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.957596 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" event={"ID":"d47e1fd3-e20b-456a-84b1-65524f29abfc","Type":"ContainerDied","Data":"619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980"} Jul 29 13:12:30.957719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.957642 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl" event={"ID":"d47e1fd3-e20b-456a-84b1-65524f29abfc","Type":"ContainerDied","Data":"1af744dd2b026fa269d5995e9363212d0f224b9006c7f652df7152a34e810523"} Jul 29 13:12:30.957719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.957664 2563 scope.go:117] "RemoveContainer" containerID="619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980" Jul 29 13:12:30.967583 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.967566 2563 scope.go:117] "RemoveContainer" containerID="1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59" Jul 29 13:12:30.975161 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.975127 2563 scope.go:117] "RemoveContainer" containerID="619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980" Jul 29 13:12:30.975397 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:30.975379 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980\": container with ID starting with 619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980 not found: ID does not exist" containerID="619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980" Jul 29 13:12:30.975485 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.975409 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980"} err="failed to get container status \"619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980\": rpc error: code = NotFound desc = could not find container \"619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980\": container with ID starting with 619b403424bd8e6b373e5d29dd534087fb825a6c6664d04a1acb1ce444355980 not found: ID does not exist" Jul 29 13:12:30.975485 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.975433 2563 scope.go:117] "RemoveContainer" containerID="1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59" Jul 29 13:12:30.975703 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:30.975689 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59\": container with ID starting with 1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59 not found: ID does not exist" containerID="1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59" Jul 29 13:12:30.975755 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.975714 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59"} err="failed to get container status \"1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59\": rpc error: code = NotFound desc = could not find container \"1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59\": container with ID starting with 1b9bb30905efa4ac5893e1e3bce439edcd76b766f3d522e0f7c217323592da59 not found: ID does not exist" Jul 29 13:12:30.981291 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.981269 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl"] Jul 29 13:12:30.987820 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:30.987799 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a7e361-openshift-default-698cc47f9b-mtlzl"] Jul 29 13:12:31.657654 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:31.657624 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" path="/var/lib/kubelet/pods/d47e1fd3-e20b-456a-84b1-65524f29abfc/volumes" Jul 29 13:12:35.844279 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:35.844246 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:35.885978 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:35.885944 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484"] Jul 29 13:12:35.886173 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:35.886151 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" podUID="0a44a66d-7047-4bf7-95ec-fbf4763bf559" containerName="payload-pre-processing" containerID="cri-o://acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8" gracePeriod=30 Jul 29 13:12:35.944530 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:35.944499 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:36.044279 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.044254 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:36.111840 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.111741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:36.145676 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.145655 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:12:36.259784 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.259757 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a44a66d-7047-4bf7-95ec-fbf4763bf559-plugins-config-volume\") pod \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " Jul 29 13:12:36.259940 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.259884 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzgvx\" (UniqueName: \"kubernetes.io/projected/0a44a66d-7047-4bf7-95ec-fbf4763bf559-kube-api-access-xzgvx\") pod \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\" (UID: \"0a44a66d-7047-4bf7-95ec-fbf4763bf559\") " Jul 29 13:12:36.260134 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.260109 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a44a66d-7047-4bf7-95ec-fbf4763bf559-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a44a66d-7047-4bf7-95ec-fbf4763bf559" (UID: "0a44a66d-7047-4bf7-95ec-fbf4763bf559"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:12:36.261819 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.261792 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a44a66d-7047-4bf7-95ec-fbf4763bf559-kube-api-access-xzgvx" (OuterVolumeSpecName: "kube-api-access-xzgvx") pod "0a44a66d-7047-4bf7-95ec-fbf4763bf559" (UID: "0a44a66d-7047-4bf7-95ec-fbf4763bf559"). InnerVolumeSpecName "kube-api-access-xzgvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:36.360709 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.360685 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzgvx\" (UniqueName: \"kubernetes.io/projected/0a44a66d-7047-4bf7-95ec-fbf4763bf559-kube-api-access-xzgvx\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:36.360709 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.360708 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a44a66d-7047-4bf7-95ec-fbf4763bf559-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:36.450360 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.450283 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:36.986757 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.986723 2563 generic.go:356] "Generic (PLEG): container finished" podID="0a44a66d-7047-4bf7-95ec-fbf4763bf559" containerID="acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8" exitCode=0 Jul 29 13:12:36.986757 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.986758 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" Jul 29 13:12:36.987253 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.986775 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" event={"ID":"0a44a66d-7047-4bf7-95ec-fbf4763bf559","Type":"ContainerDied","Data":"acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8"} Jul 29 13:12:36.987253 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.986820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484" event={"ID":"0a44a66d-7047-4bf7-95ec-fbf4763bf559","Type":"ContainerDied","Data":"685d89548fa7ad227fe8f867c3a4975cb552e2b5ecc7b7ba53274d528912e8ee"} Jul 29 13:12:36.987253 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.986843 2563 scope.go:117] "RemoveContainer" containerID="acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8" Jul 29 13:12:36.994988 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.994972 2563 scope.go:117] "RemoveContainer" containerID="acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8" Jul 29 13:12:36.995225 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:36.995206 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8\": container with ID starting with acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8 not found: ID does not exist" containerID="acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8" Jul 29 13:12:36.995289 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:36.995233 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8"} err="failed to get container status \"acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8\": rpc error: code = NotFound desc = could not find container \"acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8\": container with ID starting with acd7815be40c8bfab5d9c20b2d120e57fe69bbaebbe8e94725161c665c4e8ef8 not found: ID does not exist" Jul 29 13:12:37.010733 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:37.010670 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484"] Jul 29 13:12:37.012351 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:37.012328 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef0894-db748895b-2w484"] Jul 29 13:12:37.659447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:37.659413 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a44a66d-7047-4bf7-95ec-fbf4763bf559" path="/var/lib/kubelet/pods/0a44a66d-7047-4bf7-95ec-fbf4763bf559/volumes" Jul 29 13:12:42.624297 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:42.624266 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt"] Jul 29 13:12:42.624714 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:42.624484 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" containerID="cri-o://d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4" gracePeriod=30 Jul 29 13:12:43.090722 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.090684 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:43.191887 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.191854 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:43.292417 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.292388 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:43.575541 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575515 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8"] Jul 29 13:12:43.575834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575822 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfdd954-534e-4b5b-bf65-ef138cedebb1" containerName="payload-processing" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575836 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfdd954-534e-4b5b-bf65-ef138cedebb1" containerName="payload-processing" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575849 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d53ce5-eba9-4843-856d-563e280de8b4" containerName="payload-pre-processing" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575857 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d53ce5-eba9-4843-856d-563e280de8b4" containerName="payload-pre-processing" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575864 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575869 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575875 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a44a66d-7047-4bf7-95ec-fbf4763bf559" containerName="payload-pre-processing" Jul 29 13:12:43.575877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575880 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a44a66d-7047-4bf7-95ec-fbf4763bf559" containerName="payload-pre-processing" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575892 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575900 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575949 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575955 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a44a66d-7047-4bf7-95ec-fbf4763bf559" containerName="payload-pre-processing" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575962 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d53ce5-eba9-4843-856d-563e280de8b4" containerName="payload-pre-processing" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575968 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d47e1fd3-e20b-456a-84b1-65524f29abfc" containerName="istio-proxy" Jul 29 13:12:43.576085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.575974 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdfdd954-534e-4b5b-bf65-ef138cedebb1" containerName="payload-processing" Jul 29 13:12:43.580104 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.580085 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.584863 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.584844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e8f960-openshift-default-dockercfg-jsns7\"" Jul 29 13:12:43.597652 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.597616 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8"] Jul 29 13:12:43.619476 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.619430 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:43.724254 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724215 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-socket\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724259 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-istio-token\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724376 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-data\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/897262a3-541d-432e-aa0c-8f825a51a171-istio-podinfo\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724433 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5xk\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-kube-api-access-nj5xk\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-envoy\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-credential-socket\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-certs\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.724731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.724728 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/897262a3-541d-432e-aa0c-8f825a51a171-istiod-ca-cert\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826082 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-data\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826082 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/897262a3-541d-432e-aa0c-8f825a51a171-istio-podinfo\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826082 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5xk\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-kube-api-access-nj5xk\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-envoy\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-credential-socket\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826304 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-certs\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/897262a3-541d-432e-aa0c-8f825a51a171-istiod-ca-cert\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826596 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826409 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-socket\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826596 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826412 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-data\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826596 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826439 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-istio-token\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826822 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826630 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-certs\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.826822 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.826706 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-socket\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.827074 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.827051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/897262a3-541d-432e-aa0c-8f825a51a171-istiod-ca-cert\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.827118 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.827107 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-credential-socket\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.828624 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.828599 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-envoy\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.828899 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.828881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/897262a3-541d-432e-aa0c-8f825a51a171-istio-podinfo\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.833701 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.833678 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-istio-token\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.833836 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.833816 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5xk\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-kube-api-access-nj5xk\") pod \"e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:43.889828 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:43.889804 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:44.025795 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:44.025654 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8"] Jul 29 13:12:44.028281 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:12:44.028255 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897262a3_541d_432e_aa0c_8f825a51a171.slice/crio-92b566bddd8c3a243b6f3f4bdcc61b3e681600c4017531b90ce03aaae11beea3 WatchSource:0}: Error finding container 92b566bddd8c3a243b6f3f4bdcc61b3e681600c4017531b90ce03aaae11beea3: Status 404 returned error can't find the container with id 92b566bddd8c3a243b6f3f4bdcc61b3e681600c4017531b90ce03aaae11beea3 Jul 29 13:12:44.030620 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:44.030587 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:12:44.030746 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:44.030658 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:12:44.030746 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:44.030700 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:12:44.049582 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:44.049560 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:45.022022 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:45.021991 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" event={"ID":"897262a3-541d-432e-aa0c-8f825a51a171","Type":"ContainerStarted","Data":"14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c"} Jul 29 13:12:45.022366 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:45.022042 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" event={"ID":"897262a3-541d-432e-aa0c-8f825a51a171","Type":"ContainerStarted","Data":"92b566bddd8c3a243b6f3f4bdcc61b3e681600c4017531b90ce03aaae11beea3"} Jul 29 13:12:45.049581 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:45.049537 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" podStartSLOduration=2.049522587 podStartE2EDuration="2.049522587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:12:45.047282808 +0000 UTC m=+2837.950479483" watchObservedRunningTime="2026-07-29 13:12:45.049522587 +0000 UTC m=+2837.952719347" Jul 29 13:12:45.890145 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:45.890113 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:45.894869 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:45.894845 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:46.026382 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:46.026350 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:46.027300 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:46.027280 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:12:47.054026 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.053994 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:47.158653 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.158621 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:47.266544 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.266509 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:47.364938 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.364844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:47.752410 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.752377 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:47.853760 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.853724 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:12:47.867550 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.867523 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl"] Jul 29 13:12:47.870935 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.870920 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:47.873338 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.873318 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-e8f960-dockercfg-sd2tb\"" Jul 29 13:12:47.873433 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.873339 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-e8f960\"" Jul 29 13:12:47.882318 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.882296 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl"] Jul 29 13:12:47.884056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.884029 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:12:47.887258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.887203 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw"] Jul 29 13:12:47.887664 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.887643 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" Jul 29 13:12:47.887750 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.887667 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" Jul 29 13:12:47.887750 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.887700 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" Jul 29 13:12:47.887750 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.887708 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" Jul 29 13:12:47.887848 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.887817 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" Jul 29 13:12:47.890644 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.890626 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:47.903311 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.903291 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw"] Jul 29 13:12:47.959180 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.959151 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93c3e486-d27e-412b-b072-19ffae8aa7b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:47.959327 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:47.959309 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbb2k\" (UniqueName: \"kubernetes.io/projected/93c3e486-d27e-412b-b072-19ffae8aa7b7-kube-api-access-rbb2k\") pod \"payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:48.037989 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.037950 2563 generic.go:356] "Generic (PLEG): container finished" podID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerID="d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4" exitCode=0 Jul 29 13:12:48.038139 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.037994 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" Jul 29 13:12:48.038139 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.038025 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" event={"ID":"0044f27f-90a5-4207-bdc5-561f129c6a10","Type":"ContainerDied","Data":"d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4"} Jul 29 13:12:48.038139 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.038081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt" event={"ID":"0044f27f-90a5-4207-bdc5-561f129c6a10","Type":"ContainerDied","Data":"42871a0db4598f919c6e3cd3d2175afa4ebf212524ece894d4dfe11ade30da14"} Jul 29 13:12:48.038139 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.038112 2563 scope.go:117] "RemoveContainer" containerID="d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4" Jul 29 13:12:48.046897 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.046877 2563 scope.go:117] "RemoveContainer" containerID="5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8" Jul 29 13:12:48.055975 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.055958 2563 scope.go:117] "RemoveContainer" containerID="d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4" Jul 29 13:12:48.056230 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:48.056211 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4\": container with ID starting with d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4 not found: ID does not exist" containerID="d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4" Jul 29 13:12:48.056280 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.056239 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4"} err="failed to get container status \"d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4\": rpc error: code = NotFound desc = could not find container \"d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4\": container with ID starting with d342b0ceb565bf88257725d72452df4372b51c683b5ecd0646643a4e84e10ef4 not found: ID does not exist" Jul 29 13:12:48.056280 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.056255 2563 scope.go:117] "RemoveContainer" containerID="5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8" Jul 29 13:12:48.056523 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:12:48.056502 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8\": container with ID starting with 5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8 not found: ID does not exist" containerID="5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8" Jul 29 13:12:48.056589 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.056534 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8"} err="failed to get container status \"5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8\": rpc error: code = NotFound desc = could not find container \"5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8\": container with ID starting with 5263cebf847b1288653571478eeeef76d74ca670f77baa06b920546139f778c8 not found: ID does not exist" Jul 29 13:12:48.059803 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.059778 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-data\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.059905 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.059817 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-podinfo\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.059905 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.059842 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc9gd\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-kube-api-access-rc9gd\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.059905 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.059878 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0044f27f-90a5-4207-bdc5-561f129c6a10-istiod-ca-cert\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.060056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.059938 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-envoy\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.060056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.059994 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-certs\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.060056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060031 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-socket\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.060213 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060062 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-token\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.060213 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060088 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-credential-socket\") pod \"0044f27f-90a5-4207-bdc5-561f129c6a10\" (UID: \"0044f27f-90a5-4207-bdc5-561f129c6a10\") " Jul 29 13:12:48.060320 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060254 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93c3e486-d27e-412b-b072-19ffae8aa7b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:48.060320 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060295 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:48.060425 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060317 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0044f27f-90a5-4207-bdc5-561f129c6a10-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:12:48.060425 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060364 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtd7c\" (UniqueName: \"kubernetes.io/projected/dc16b25c-02d3-4aea-8348-568911772572-kube-api-access-xtd7c\") pod \"payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.060554 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbb2k\" (UniqueName: \"kubernetes.io/projected/93c3e486-d27e-412b-b072-19ffae8aa7b7-kube-api-access-rbb2k\") pod \"payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:48.060554 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060547 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc16b25c-02d3-4aea-8348-568911772572-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.060659 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060609 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-data" (OuterVolumeSpecName: "istio-data") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:48.060659 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060630 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.060902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060866 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0044f27f-90a5-4207-bdc5-561f129c6a10-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.060902 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060870 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:48.061074 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.060924 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:48.061453 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.061430 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93c3e486-d27e-412b-b072-19ffae8aa7b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:48.062327 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.062283 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:12:48.062441 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.062324 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-kube-api-access-rc9gd" (OuterVolumeSpecName: "kube-api-access-rc9gd") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "kube-api-access-rc9gd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:48.062729 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.062650 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:12:48.062729 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.062665 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-token" (OuterVolumeSpecName: "istio-token") pod "0044f27f-90a5-4207-bdc5-561f129c6a10" (UID: "0044f27f-90a5-4207-bdc5-561f129c6a10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:12:48.069280 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.069142 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbb2k\" (UniqueName: \"kubernetes.io/projected/93c3e486-d27e-412b-b072-19ffae8aa7b7-kube-api-access-rbb2k\") pod \"payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:48.161561 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161499 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtd7c\" (UniqueName: \"kubernetes.io/projected/dc16b25c-02d3-4aea-8348-568911772572-kube-api-access-xtd7c\") pod \"payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.161655 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161604 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc16b25c-02d3-4aea-8348-568911772572-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.161714 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161697 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.161765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161715 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.161765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161728 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.161765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161742 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.161765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161756 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.161944 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161777 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0044f27f-90a5-4207-bdc5-561f129c6a10-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.161944 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.161791 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc9gd\" (UniqueName: \"kubernetes.io/projected/0044f27f-90a5-4207-bdc5-561f129c6a10-kube-api-access-rc9gd\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:12:48.162358 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.162340 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc16b25c-02d3-4aea-8348-568911772572-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.170341 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.170321 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtd7c\" (UniqueName: \"kubernetes.io/projected/dc16b25c-02d3-4aea-8348-568911772572-kube-api-access-xtd7c\") pod \"payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.190803 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.190785 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:48.199841 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.199822 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:48.338797 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.338767 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl"] Jul 29 13:12:48.341030 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:12:48.341005 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c3e486_d27e_412b_b072_19ffae8aa7b7.slice/crio-01c5d7a7cb943ad6497f8434eb53b1858492fed5a7f6ea3a0bfe45dbbbc5e903 WatchSource:0}: Error finding container 01c5d7a7cb943ad6497f8434eb53b1858492fed5a7f6ea3a0bfe45dbbbc5e903: Status 404 returned error can't find the container with id 01c5d7a7cb943ad6497f8434eb53b1858492fed5a7f6ea3a0bfe45dbbbc5e903 Jul 29 13:12:48.362152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.362129 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt"] Jul 29 13:12:48.365921 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.365899 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ef0894-openshift-default-6cf896c948-mktlt"] Jul 29 13:12:48.566523 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:48.566498 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw"] Jul 29 13:12:48.569402 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:12:48.569367 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc16b25c_02d3_4aea_8348_568911772572.slice/crio-aa00f6ee8b74b0aa8300f1b0e62ac9e15fc458ea12cbff3cb1f2509470da3fb3 WatchSource:0}: Error finding container aa00f6ee8b74b0aa8300f1b0e62ac9e15fc458ea12cbff3cb1f2509470da3fb3: Status 404 returned error can't find the container with id aa00f6ee8b74b0aa8300f1b0e62ac9e15fc458ea12cbff3cb1f2509470da3fb3 Jul 29 13:12:49.044887 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.044849 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" event={"ID":"dc16b25c-02d3-4aea-8348-568911772572","Type":"ContainerStarted","Data":"93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda"} Jul 29 13:12:49.045064 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.044934 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" event={"ID":"dc16b25c-02d3-4aea-8348-568911772572","Type":"ContainerStarted","Data":"aa00f6ee8b74b0aa8300f1b0e62ac9e15fc458ea12cbff3cb1f2509470da3fb3"} Jul 29 13:12:49.045064 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.044961 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:12:49.046968 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.046945 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" event={"ID":"93c3e486-d27e-412b-b072-19ffae8aa7b7","Type":"ContainerStarted","Data":"41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8"} Jul 29 13:12:49.047087 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.046980 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" event={"ID":"93c3e486-d27e-412b-b072-19ffae8aa7b7","Type":"ContainerStarted","Data":"01c5d7a7cb943ad6497f8434eb53b1858492fed5a7f6ea3a0bfe45dbbbc5e903"} Jul 29 13:12:49.047087 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.047018 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:12:49.063545 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.063496 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" podStartSLOduration=2.06348093 podStartE2EDuration="2.06348093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:12:49.062668715 +0000 UTC m=+2841.965865392" watchObservedRunningTime="2026-07-29 13:12:49.06348093 +0000 UTC m=+2841.966677603" Jul 29 13:12:49.082229 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.082192 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" podStartSLOduration=2.082179308 podStartE2EDuration="2.082179308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:12:49.079739323 +0000 UTC m=+2841.982935999" watchObservedRunningTime="2026-07-29 13:12:49.082179308 +0000 UTC m=+2841.985375990" Jul 29 13:12:49.658909 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:12:49.658868 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" path="/var/lib/kubelet/pods/0044f27f-90a5-4207-bdc5-561f129c6a10/volumes" Jul 29 13:13:00.054204 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:00.054163 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:13:00.054591 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:00.054258 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:13:01.028336 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.028282 2563 patch_prober.go:28] interesting pod/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 29 13:13:01.028516 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.028333 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 29 13:13:01.111127 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.111098 2563 generic.go:356] "Generic (PLEG): container finished" podID="897262a3-541d-432e-aa0c-8f825a51a171" containerID="14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c" exitCode=0 Jul 29 13:13:01.111424 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.111170 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" event={"ID":"897262a3-541d-432e-aa0c-8f825a51a171","Type":"ContainerDied","Data":"14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c"} Jul 29 13:13:01.111539 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.111517 2563 scope.go:117] "RemoveContainer" containerID="14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c" Jul 29 13:13:01.112405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.112381 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:01.112481 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.112447 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:01.112530 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:01.112497 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:02.117180 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:02.117144 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" event={"ID":"897262a3-541d-432e-aa0c-8f825a51a171","Type":"ContainerStarted","Data":"e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755"} Jul 29 13:13:02.890090 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:02.890045 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:13:02.890367 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:02.890338 2563 patch_prober.go:28] interesting pod/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 29 13:13:02.890446 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:02.890404 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 29 13:13:03.890683 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:03.890651 2563 patch_prober.go:28] interesting pod/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 29 13:13:03.891056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:03.890710 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 29 13:13:04.890770 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:04.890736 2563 patch_prober.go:28] interesting pod/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 29 13:13:04.891163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:04.890788 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 29 13:13:05.894510 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:05.894482 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:13:05.894962 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:05.894858 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:13:05.895398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:05.895379 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:13:12.067148 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.067100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:12.559474 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.559406 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:12.860813 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.860731 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph"] Jul 29 13:13:12.861166 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.861149 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0044f27f-90a5-4207-bdc5-561f129c6a10" containerName="istio-proxy" Jul 29 13:13:12.864541 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.864519 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.868055 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.867607 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-2f4f5e-openshift-default-dockercfg-sjqnt\"" Jul 29 13:13:12.880133 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.880001 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph"] Jul 29 13:13:12.891199 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.891171 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:12.972929 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.972892 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-credential-socket\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973184 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-token\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973313 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973297 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-podinfo\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973519 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-certs\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973519 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973445 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-envoy\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973660 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973524 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsgj8\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-kube-api-access-nsgj8\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973660 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-data\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973660 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-socket\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:12.973660 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:12.973647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istiod-ca-cert\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074443 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074414 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsgj8\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-kube-api-access-nsgj8\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074450 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-data\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074531 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-socket\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074578 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istiod-ca-cert\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074639 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-credential-socket\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074676 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-token\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074704 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-podinfo\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074755 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-certs\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.074834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074778 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-envoy\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.075258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-socket\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.075258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.074922 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-credential-socket\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.075258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.075136 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-data\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.075258 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.075166 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-certs\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.075404 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.075348 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istiod-ca-cert\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.076995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.076969 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-envoy\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.077360 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.077338 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-podinfo\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.083051 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.083018 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-token\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.083189 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.083172 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsgj8\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-kube-api-access-nsgj8\") pod \"e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.178119 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.178071 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:13.323183 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:13:13.323154 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3b8eb6_a9d5_49a1_ba21_50f13e70315c.slice/crio-923de6b741d5c8099ccc49ef03c22f00f8627b6dc70ce2f526434e11919def08 WatchSource:0}: Error finding container 923de6b741d5c8099ccc49ef03c22f00f8627b6dc70ce2f526434e11919def08: Status 404 returned error can't find the container with id 923de6b741d5c8099ccc49ef03c22f00f8627b6dc70ce2f526434e11919def08 Jul 29 13:13:13.323337 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.323312 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph"] Jul 29 13:13:13.325654 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.325616 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:13.325743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.325693 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:13.325743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.325733 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:13.517891 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:13.517853 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:14.173328 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:14.173294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" event={"ID":"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c","Type":"ContainerStarted","Data":"00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6"} Jul 29 13:13:14.173707 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:14.173354 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" event={"ID":"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c","Type":"ContainerStarted","Data":"923de6b741d5c8099ccc49ef03c22f00f8627b6dc70ce2f526434e11919def08"} Jul 29 13:13:14.178249 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:14.178220 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:14.182954 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:14.182931 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:14.198143 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:14.198093 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" podStartSLOduration=2.198075963 podStartE2EDuration="2.198075963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:13:14.194981527 +0000 UTC m=+2867.098178202" watchObservedRunningTime="2026-07-29 13:13:14.198075963 +0000 UTC m=+2867.101272640" Jul 29 13:13:15.178868 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:15.178839 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:15.179838 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:15.179819 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:16.236318 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:16.236284 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:16.332887 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:16.332847 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:16.519108 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:16.519012 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:16.835914 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:16.835830 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:16.936194 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:16.936159 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:17.051636 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.051606 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm"] Jul 29 13:13:17.055018 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.055003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.057567 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.057547 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2f4f5e\"" Jul 29 13:13:17.057702 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.057686 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2f4f5e-dockercfg-ncqp6\"" Jul 29 13:13:17.064306 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.064286 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm"] Jul 29 13:13:17.109129 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.109022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.109129 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.109124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nstbl\" (UniqueName: \"kubernetes.io/projected/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-kube-api-access-nstbl\") pod \"payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.210237 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.210204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nstbl\" (UniqueName: \"kubernetes.io/projected/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-kube-api-access-nstbl\") pod \"payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.210400 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.210307 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.210972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.210952 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.218759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.218738 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nstbl\" (UniqueName: \"kubernetes.io/projected/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-kube-api-access-nstbl\") pod \"payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.367314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.367249 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:17.512754 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:17.512725 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm"] Jul 29 13:13:17.515107 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:13:17.515079 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ea1f2b_8fab_4e62_89d9_8d77d885f6aa.slice/crio-19dd0ff52808677faa0ba9a43245ba7967c92174e798f5208a67ce1a99eaa615 WatchSource:0}: Error finding container 19dd0ff52808677faa0ba9a43245ba7967c92174e798f5208a67ce1a99eaa615: Status 404 returned error can't find the container with id 19dd0ff52808677faa0ba9a43245ba7967c92174e798f5208a67ce1a99eaa615 Jul 29 13:13:18.195651 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:18.195609 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" event={"ID":"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa","Type":"ContainerStarted","Data":"657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a"} Jul 29 13:13:18.195816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:18.195666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" event={"ID":"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa","Type":"ContainerStarted","Data":"19dd0ff52808677faa0ba9a43245ba7967c92174e798f5208a67ce1a99eaa615"} Jul 29 13:13:18.195816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:18.195713 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:18.211790 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:18.211741 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" podStartSLOduration=1.211728156 podStartE2EDuration="1.211728156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:13:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:13:18.21152972 +0000 UTC m=+2871.114726396" watchObservedRunningTime="2026-07-29 13:13:18.211728156 +0000 UTC m=+2871.114924834" Jul 29 13:13:29.202526 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:29.202492 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:13:30.477504 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:30.477446 2563 patch_prober.go:28] interesting pod/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:56896->10.134.0.61:15021: read: connection reset by peer" start-of-body= Jul 29 13:13:30.477853 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:30.477540 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:56896->10.134.0.61:15021: read: connection reset by peer" Jul 29 13:13:31.258018 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:31.257982 2563 generic.go:356] "Generic (PLEG): container finished" podID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerID="00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6" exitCode=0 Jul 29 13:13:31.258191 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:31.258063 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" event={"ID":"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c","Type":"ContainerDied","Data":"00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6"} Jul 29 13:13:31.258452 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:31.258438 2563 scope.go:117] "RemoveContainer" containerID="00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6" Jul 29 13:13:31.259506 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:31.259475 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:31.259607 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:31.259558 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:31.259670 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:31.259596 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:13:32.265582 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:32.265539 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" event={"ID":"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c","Type":"ContainerStarted","Data":"454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df"} Jul 29 13:13:33.179429 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:33.178955 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:33.179429 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:33.179104 2563 patch_prober.go:28] interesting pod/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 29 13:13:33.179429 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:33.179160 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 29 13:13:34.179414 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:34.178974 2563 patch_prober.go:28] interesting pod/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 29 13:13:34.179414 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:34.179035 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 29 13:13:35.179576 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:35.179115 2563 patch_prober.go:28] interesting pod/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 29 13:13:35.179576 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:35.179191 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 29 13:13:36.182064 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:36.182036 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:36.182417 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:36.182385 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:36.183010 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:36.182994 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:13:41.283019 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:41.282982 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:59.965341 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:59.965299 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:13:59.983055 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:59.983025 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw"] Jul 29 13:13:59.983326 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:59.983286 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" podUID="dc16b25c-02d3-4aea-8348-568911772572" containerName="payload-processing" containerID="cri-o://93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda" gracePeriod=30 Jul 29 13:13:59.992945 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:59.992762 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl"] Jul 29 13:13:59.993048 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:13:59.992972 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" podUID="93c3e486-d27e-412b-b072-19ffae8aa7b7" containerName="payload-pre-processing" containerID="cri-o://41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8" gracePeriod=30 Jul 29 13:14:00.053395 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.053357 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" podUID="dc16b25c-02d3-4aea-8348-568911772572" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.60:9004: connect: connection refused" Jul 29 13:14:00.053536 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.053399 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" podUID="93c3e486-d27e-412b-b072-19ffae8aa7b7" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.59:9004: connect: connection refused" Jul 29 13:14:00.067655 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.067626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:00.248022 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.247998 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:14:00.275254 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.275233 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:14:00.278394 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.278375 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtd7c\" (UniqueName: \"kubernetes.io/projected/dc16b25c-02d3-4aea-8348-568911772572-kube-api-access-xtd7c\") pod \"dc16b25c-02d3-4aea-8348-568911772572\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " Jul 29 13:14:00.278506 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.278421 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc16b25c-02d3-4aea-8348-568911772572-plugins-config-volume\") pod \"dc16b25c-02d3-4aea-8348-568911772572\" (UID: \"dc16b25c-02d3-4aea-8348-568911772572\") " Jul 29 13:14:00.278840 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.278819 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc16b25c-02d3-4aea-8348-568911772572-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc16b25c-02d3-4aea-8348-568911772572" (UID: "dc16b25c-02d3-4aea-8348-568911772572"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:14:00.280317 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.280298 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc16b25c-02d3-4aea-8348-568911772572-kube-api-access-xtd7c" (OuterVolumeSpecName: "kube-api-access-xtd7c") pod "dc16b25c-02d3-4aea-8348-568911772572" (UID: "dc16b25c-02d3-4aea-8348-568911772572"). InnerVolumeSpecName "kube-api-access-xtd7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:00.285931 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.285900 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:00.344695 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.344653 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:00.379768 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.379743 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbb2k\" (UniqueName: \"kubernetes.io/projected/93c3e486-d27e-412b-b072-19ffae8aa7b7-kube-api-access-rbb2k\") pod \"93c3e486-d27e-412b-b072-19ffae8aa7b7\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " Jul 29 13:14:00.379881 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.379810 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93c3e486-d27e-412b-b072-19ffae8aa7b7-plugins-config-volume\") pod \"93c3e486-d27e-412b-b072-19ffae8aa7b7\" (UID: \"93c3e486-d27e-412b-b072-19ffae8aa7b7\") " Jul 29 13:14:00.380068 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.380048 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtd7c\" (UniqueName: \"kubernetes.io/projected/dc16b25c-02d3-4aea-8348-568911772572-kube-api-access-xtd7c\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:00.380138 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.380100 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc16b25c-02d3-4aea-8348-568911772572-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:00.380138 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.380117 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c3e486-d27e-412b-b072-19ffae8aa7b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "93c3e486-d27e-412b-b072-19ffae8aa7b7" (UID: "93c3e486-d27e-412b-b072-19ffae8aa7b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:14:00.381695 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.381677 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c3e486-d27e-412b-b072-19ffae8aa7b7-kube-api-access-rbb2k" (OuterVolumeSpecName: "kube-api-access-rbb2k") pod "93c3e486-d27e-412b-b072-19ffae8aa7b7" (UID: "93c3e486-d27e-412b-b072-19ffae8aa7b7"). InnerVolumeSpecName "kube-api-access-rbb2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:00.403151 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.403118 2563 generic.go:356] "Generic (PLEG): container finished" podID="dc16b25c-02d3-4aea-8348-568911772572" containerID="93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda" exitCode=0 Jul 29 13:14:00.403240 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.403161 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" Jul 29 13:14:00.403240 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.403200 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" event={"ID":"dc16b25c-02d3-4aea-8348-568911772572","Type":"ContainerDied","Data":"93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda"} Jul 29 13:14:00.403340 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.403246 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw" event={"ID":"dc16b25c-02d3-4aea-8348-568911772572","Type":"ContainerDied","Data":"aa00f6ee8b74b0aa8300f1b0e62ac9e15fc458ea12cbff3cb1f2509470da3fb3"} Jul 29 13:14:00.403340 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.403274 2563 scope.go:117] "RemoveContainer" containerID="93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda" Jul 29 13:14:00.404448 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.404424 2563 generic.go:356] "Generic (PLEG): container finished" podID="93c3e486-d27e-412b-b072-19ffae8aa7b7" containerID="41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8" exitCode=0 Jul 29 13:14:00.404570 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.404486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" event={"ID":"93c3e486-d27e-412b-b072-19ffae8aa7b7","Type":"ContainerDied","Data":"41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8"} Jul 29 13:14:00.404570 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.404454 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" Jul 29 13:14:00.404570 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.404527 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl" event={"ID":"93c3e486-d27e-412b-b072-19ffae8aa7b7","Type":"ContainerDied","Data":"01c5d7a7cb943ad6497f8434eb53b1858492fed5a7f6ea3a0bfe45dbbbc5e903"} Jul 29 13:14:00.412760 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.412743 2563 scope.go:117] "RemoveContainer" containerID="93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda" Jul 29 13:14:00.413020 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:00.413002 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda\": container with ID starting with 93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda not found: ID does not exist" containerID="93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda" Jul 29 13:14:00.413108 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.413039 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda"} err="failed to get container status \"93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda\": rpc error: code = NotFound desc = could not find container \"93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda\": container with ID starting with 93e1d393a77ecc216c6e1737df596b3deb21b2f3c25524b94899b8175da4ebda not found: ID does not exist" Jul 29 13:14:00.413108 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.413061 2563 scope.go:117] "RemoveContainer" containerID="41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8" Jul 29 13:14:00.420543 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.420522 2563 scope.go:117] "RemoveContainer" containerID="41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8" Jul 29 13:14:00.420773 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:00.420755 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8\": container with ID starting with 41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8 not found: ID does not exist" containerID="41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8" Jul 29 13:14:00.420828 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.420778 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8"} err="failed to get container status \"41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8\": rpc error: code = NotFound desc = could not find container \"41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8\": container with ID starting with 41d4174be3c9273d52f7e487c988e02819deb81633a9912775fd39940ea9d4b8 not found: ID does not exist" Jul 29 13:14:00.427808 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.427787 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl"] Jul 29 13:14:00.431979 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.431961 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-e8f960-764848bbdd-lrtvl"] Jul 29 13:14:00.441765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.441747 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw"] Jul 29 13:14:00.445421 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.445402 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e8f960-6f6df5bdf6-kwjdw"] Jul 29 13:14:00.481321 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.481273 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbb2k\" (UniqueName: \"kubernetes.io/projected/93c3e486-d27e-412b-b072-19ffae8aa7b7-kube-api-access-rbb2k\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:00.481321 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.481293 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/93c3e486-d27e-412b-b072-19ffae8aa7b7-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:00.528988 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:00.528958 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:01.658385 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:01.658350 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93c3e486-d27e-412b-b072-19ffae8aa7b7" path="/var/lib/kubelet/pods/93c3e486-d27e-412b-b072-19ffae8aa7b7/volumes" Jul 29 13:14:01.658776 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:01.658724 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc16b25c-02d3-4aea-8348-568911772572" path="/var/lib/kubelet/pods/dc16b25c-02d3-4aea-8348-568911772572/volumes" Jul 29 13:14:06.272724 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:06.272685 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:06.373297 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:06.373259 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:06.585894 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:06.585820 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8"] Jul 29 13:14:06.586063 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:06.586035 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" containerID="cri-o://e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755" gracePeriod=30 Jul 29 13:14:07.003813 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:07.003769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:07.100753 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:07.100719 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:11.829565 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.829540 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:14:11.878730 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878661 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/897262a3-541d-432e-aa0c-8f825a51a171-istio-podinfo\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.878856 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878726 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj5xk\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-kube-api-access-nj5xk\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.878856 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878787 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-certs\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.878856 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878835 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-socket\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.879017 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878860 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-envoy\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.879017 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878892 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/897262a3-541d-432e-aa0c-8f825a51a171-istiod-ca-cert\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.879017 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878918 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-istio-token\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.879017 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878945 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-data\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.879017 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.878987 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-credential-socket\") pod \"897262a3-541d-432e-aa0c-8f825a51a171\" (UID: \"897262a3-541d-432e-aa0c-8f825a51a171\") " Jul 29 13:14:11.879270 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.879040 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:11.879377 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.879333 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:11.879377 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.879337 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.879570 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.879381 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:11.879656 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.879634 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/897262a3-541d-432e-aa0c-8f825a51a171-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:14:11.880024 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.880000 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-data" (OuterVolumeSpecName: "istio-data") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:11.881260 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.881229 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:11.881331 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.881301 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/897262a3-541d-432e-aa0c-8f825a51a171-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:14:11.881331 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.881304 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-kube-api-access-nj5xk" (OuterVolumeSpecName: "kube-api-access-nj5xk") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "kube-api-access-nj5xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:11.881512 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.881484 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-istio-token" (OuterVolumeSpecName: "istio-token") pod "897262a3-541d-432e-aa0c-8f825a51a171" (UID: "897262a3-541d-432e-aa0c-8f825a51a171"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:11.980358 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980336 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980358 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980355 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980364 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/897262a3-541d-432e-aa0c-8f825a51a171-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980373 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980380 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980387 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/897262a3-541d-432e-aa0c-8f825a51a171-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980396 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/897262a3-541d-432e-aa0c-8f825a51a171-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:11.980501 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:11.980405 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj5xk\" (UniqueName: \"kubernetes.io/projected/897262a3-541d-432e-aa0c-8f825a51a171-kube-api-access-nj5xk\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:12.467306 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.467272 2563 generic.go:356] "Generic (PLEG): container finished" podID="897262a3-541d-432e-aa0c-8f825a51a171" containerID="e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755" exitCode=0 Jul 29 13:14:12.467566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.467318 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" Jul 29 13:14:12.467566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.467350 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" event={"ID":"897262a3-541d-432e-aa0c-8f825a51a171","Type":"ContainerDied","Data":"e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755"} Jul 29 13:14:12.467566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.467412 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8" event={"ID":"897262a3-541d-432e-aa0c-8f825a51a171","Type":"ContainerDied","Data":"92b566bddd8c3a243b6f3f4bdcc61b3e681600c4017531b90ce03aaae11beea3"} Jul 29 13:14:12.467566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.467434 2563 scope.go:117] "RemoveContainer" containerID="e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755" Jul 29 13:14:12.477214 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.477196 2563 scope.go:117] "RemoveContainer" containerID="14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c" Jul 29 13:14:12.484915 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.484897 2563 scope.go:117] "RemoveContainer" containerID="e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755" Jul 29 13:14:12.485166 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:12.485148 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755\": container with ID starting with e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755 not found: ID does not exist" containerID="e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755" Jul 29 13:14:12.485225 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.485175 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755"} err="failed to get container status \"e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755\": rpc error: code = NotFound desc = could not find container \"e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755\": container with ID starting with e6c3dc81358361d361303c3e6e7f6cf20d213459d819d1f0954a12121a55a755 not found: ID does not exist" Jul 29 13:14:12.485225 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.485192 2563 scope.go:117] "RemoveContainer" containerID="14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c" Jul 29 13:14:12.485414 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:12.485400 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c\": container with ID starting with 14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c not found: ID does not exist" containerID="14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c" Jul 29 13:14:12.485450 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.485420 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c"} err="failed to get container status \"14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c\": rpc error: code = NotFound desc = could not find container \"14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c\": container with ID starting with 14d0a268284aeb4aba9aa8dc1121c21af7f2a63361f8f8fb34d66759ad7e6b6c not found: ID does not exist" Jul 29 13:14:12.489974 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.489955 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8"] Jul 29 13:14:12.493991 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:12.493972 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e8f960-openshift-default-747b7dcb6d-xd9f8"] Jul 29 13:14:13.664831 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:13.664798 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897262a3-541d-432e-aa0c-8f825a51a171" path="/var/lib/kubelet/pods/897262a3-541d-432e-aa0c-8f825a51a171/volumes" Jul 29 13:14:22.568597 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.568559 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:22.603208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.603181 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm"] Jul 29 13:14:22.603427 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.603402 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" podUID="96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" containerName="payload-pre-processing" containerID="cri-o://657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a" gracePeriod=30 Jul 29 13:14:22.631172 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.631147 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:22.690375 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.690339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:22.750078 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.749898 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:22.850863 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.850841 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:14:22.969751 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.969720 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nstbl\" (UniqueName: \"kubernetes.io/projected/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-kube-api-access-nstbl\") pod \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " Jul 29 13:14:22.969911 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.969773 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-plugins-config-volume\") pod \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\" (UID: \"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa\") " Jul 29 13:14:22.970112 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.970090 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" (UID: "96ea1f2b-8fab-4e62-89d9-8d77d885f6aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:14:22.971801 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:22.971782 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-kube-api-access-nstbl" (OuterVolumeSpecName: "kube-api-access-nstbl") pod "96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" (UID: "96ea1f2b-8fab-4e62-89d9-8d77d885f6aa"). InnerVolumeSpecName "kube-api-access-nstbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:23.070245 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.070223 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nstbl\" (UniqueName: \"kubernetes.io/projected/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-kube-api-access-nstbl\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:23.070245 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.070244 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:23.517759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.517722 2563 generic.go:356] "Generic (PLEG): container finished" podID="96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" containerID="657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a" exitCode=0 Jul 29 13:14:23.517945 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.517772 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" Jul 29 13:14:23.517945 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.517808 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" event={"ID":"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa","Type":"ContainerDied","Data":"657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a"} Jul 29 13:14:23.517945 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.517865 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm" event={"ID":"96ea1f2b-8fab-4e62-89d9-8d77d885f6aa","Type":"ContainerDied","Data":"19dd0ff52808677faa0ba9a43245ba7967c92174e798f5208a67ce1a99eaa615"} Jul 29 13:14:23.517945 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.517892 2563 scope.go:117] "RemoveContainer" containerID="657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a" Jul 29 13:14:23.525761 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.525741 2563 scope.go:117] "RemoveContainer" containerID="657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a" Jul 29 13:14:23.525992 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:23.525974 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a\": container with ID starting with 657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a not found: ID does not exist" containerID="657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a" Jul 29 13:14:23.526059 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.526004 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a"} err="failed to get container status \"657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a\": rpc error: code = NotFound desc = could not find container \"657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a\": container with ID starting with 657e12e3ac1cc72ceb62a015bd49c337c91570f16afbc7e347206f258536a97a not found: ID does not exist" Jul 29 13:14:23.539344 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.539322 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm"] Jul 29 13:14:23.543177 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.543157 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-2f4f5e-798bbcdb4d-v2vqm"] Jul 29 13:14:23.624908 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.624882 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:23.658386 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:23.658361 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" path="/var/lib/kubelet/pods/96ea1f2b-8fab-4e62-89d9-8d77d885f6aa/volumes" Jul 29 13:14:29.005815 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:29.005776 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:29.229594 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:29.229564 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph"] Jul 29 13:14:29.229830 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:29.229805 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" containerID="cri-o://454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df" gracePeriod=30 Jul 29 13:14:30.110329 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.110263 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:30.167716 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.167684 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82"] Jul 29 13:14:30.168078 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168062 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93c3e486-d27e-412b-b072-19ffae8aa7b7" containerName="payload-pre-processing" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168081 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="93c3e486-d27e-412b-b072-19ffae8aa7b7" containerName="payload-pre-processing" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168094 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" containerName="payload-pre-processing" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168104 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" containerName="payload-pre-processing" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168119 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc16b25c-02d3-4aea-8348-568911772572" containerName="payload-processing" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168128 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc16b25c-02d3-4aea-8348-568911772572" containerName="payload-processing" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168137 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168145 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" Jul 29 13:14:30.168163 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168162 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" Jul 29 13:14:30.168574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168170 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" Jul 29 13:14:30.168574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168256 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" Jul 29 13:14:30.168574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168266 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="897262a3-541d-432e-aa0c-8f825a51a171" containerName="istio-proxy" Jul 29 13:14:30.168574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168274 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ea1f2b-8fab-4e62-89d9-8d77d885f6aa" containerName="payload-pre-processing" Jul 29 13:14:30.168574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168282 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc16b25c-02d3-4aea-8348-568911772572" containerName="payload-processing" Jul 29 13:14:30.168574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.168297 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="93c3e486-d27e-412b-b072-19ffae8aa7b7" containerName="payload-pre-processing" Jul 29 13:14:30.173114 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.173094 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.175590 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.175570 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f0ad55-openshift-default-dockercfg-8s4xf\"" Jul 29 13:14:30.183984 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.183963 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82"] Jul 29 13:14:30.258393 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.258354 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:30.329061 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329034 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/936be956-5115-4a68-be8e-95b6c378a023-istiod-ca-cert\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329217 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329073 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-credential-socket\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329217 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329100 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r24ps\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-kube-api-access-r24ps\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329217 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329159 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-data\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329217 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-envoy\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329217 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329210 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-socket\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329515 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329247 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/936be956-5115-4a68-be8e-95b6c378a023-istio-podinfo\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329515 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-istio-token\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.329515 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.329332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-certs\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430223 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430145 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/936be956-5115-4a68-be8e-95b6c378a023-istio-podinfo\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430223 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430197 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-istio-token\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430408 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430306 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-certs\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430774 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/936be956-5115-4a68-be8e-95b6c378a023-istiod-ca-cert\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430843 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-credential-socket\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430843 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r24ps\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-kube-api-access-r24ps\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430946 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430856 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-data\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430946 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430880 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-envoy\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.430946 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-socket\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.431091 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.430978 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-certs\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.431151 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.431098 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-credential-socket\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.431380 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.431352 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-data\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.431535 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.431499 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/936be956-5115-4a68-be8e-95b6c378a023-istiod-ca-cert\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.431612 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.431530 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-socket\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.433130 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.433105 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/936be956-5115-4a68-be8e-95b6c378a023-istio-podinfo\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.433333 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.433312 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-envoy\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.438597 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.438574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-istio-token\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.438787 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.438767 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r24ps\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-kube-api-access-r24ps\") pod \"e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.483467 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.483442 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:30.622309 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.622282 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82"] Jul 29 13:14:30.624636 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:14:30.624603 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936be956_5115_4a68_be8e_95b6c378a023.slice/crio-dd0e3d733468aff94195ff3198393e36a91c2f196fb1957fee4096c9363b8edf WatchSource:0}: Error finding container dd0e3d733468aff94195ff3198393e36a91c2f196fb1957fee4096c9363b8edf: Status 404 returned error can't find the container with id dd0e3d733468aff94195ff3198393e36a91c2f196fb1957fee4096c9363b8edf Jul 29 13:14:30.626819 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.626786 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:30.626938 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.626865 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:30.626938 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:30.626906 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:31.088273 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:31.088237 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:31.561095 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:31.561054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" event={"ID":"936be956-5115-4a68-be8e-95b6c378a023","Type":"ContainerStarted","Data":"c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888"} Jul 29 13:14:31.561493 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:31.561121 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" event={"ID":"936be956-5115-4a68-be8e-95b6c378a023","Type":"ContainerStarted","Data":"dd0e3d733468aff94195ff3198393e36a91c2f196fb1957fee4096c9363b8edf"} Jul 29 13:14:31.582024 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:31.581978 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podStartSLOduration=1.58196197 podStartE2EDuration="1.58196197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:14:31.580554935 +0000 UTC m=+2944.483751611" watchObservedRunningTime="2026-07-29 13:14:31.58196197 +0000 UTC m=+2944.485158709" Jul 29 13:14:32.483644 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:32.483614 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:32.488009 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:32.487988 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:32.565329 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:32.565303 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:32.565959 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:32.565941 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:33.559789 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:33.559742 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:33.655077 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:33.655043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:33.755931 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:33.755889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:33.994499 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:33.994452 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:34.390965 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.390935 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:34.474443 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.474417 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:14:34.491303 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.491281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:34.527418 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527356 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn"] Jul 29 13:14:34.527712 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527697 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" Jul 29 13:14:34.527759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527714 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" Jul 29 13:14:34.527759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527737 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" Jul 29 13:14:34.527759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527743 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" Jul 29 13:14:34.527857 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527799 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" Jul 29 13:14:34.527857 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.527808 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerName="istio-proxy" Jul 29 13:14:34.530687 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.530672 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.533301 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.533279 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-f0ad55\"" Jul 29 13:14:34.533422 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.533350 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-f0ad55-dockercfg-df67p\"" Jul 29 13:14:34.541489 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.541469 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn"] Jul 29 13:14:34.569542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569517 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-data\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569642 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569556 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsgj8\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-kube-api-access-nsgj8\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569642 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569578 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istiod-ca-cert\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569642 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569606 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-podinfo\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569866 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569718 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-credential-socket\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569866 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569807 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-token\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569866 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569826 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-envoy\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.569866 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569843 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-certs\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.570065 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569877 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-socket\") pod \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\" (UID: \"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c\") " Jul 29 13:14:34.570065 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.569942 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:14:34.570159 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.570087 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:34.570305 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.570228 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.570305 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.570280 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.570305 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.570289 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:34.570565 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.570430 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:34.570565 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.570541 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:34.571927 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.571905 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:14:34.572040 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.571995 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:34.572178 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.572161 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:14:34.572249 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.572162 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-kube-api-access-nsgj8" (OuterVolumeSpecName: "kube-api-access-nsgj8") pod "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" (UID: "4a3b8eb6-a9d5-49a1-ba21-50f13e70315c"). InnerVolumeSpecName "kube-api-access-nsgj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:14:34.576191 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.576170 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k"] Jul 29 13:14:34.577954 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.577929 2563 generic.go:356] "Generic (PLEG): container finished" podID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" containerID="454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df" exitCode=0 Jul 29 13:14:34.578074 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.578061 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" Jul 29 13:14:34.580743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.580723 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" event={"ID":"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c","Type":"ContainerDied","Data":"454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df"} Jul 29 13:14:34.580832 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.580756 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph" event={"ID":"4a3b8eb6-a9d5-49a1-ba21-50f13e70315c","Type":"ContainerDied","Data":"923de6b741d5c8099ccc49ef03c22f00f8627b6dc70ce2f526434e11919def08"} Jul 29 13:14:34.580832 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.580778 2563 scope.go:117] "RemoveContainer" containerID="454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df" Jul 29 13:14:34.580965 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.580837 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.589750 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.589727 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k"] Jul 29 13:14:34.595566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.595541 2563 scope.go:117] "RemoveContainer" containerID="00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6" Jul 29 13:14:34.605552 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.605535 2563 scope.go:117] "RemoveContainer" containerID="454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df" Jul 29 13:14:34.605799 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:34.605782 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df\": container with ID starting with 454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df not found: ID does not exist" containerID="454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df" Jul 29 13:14:34.605859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.605806 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df"} err="failed to get container status \"454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df\": rpc error: code = NotFound desc = could not find container \"454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df\": container with ID starting with 454ec97f257c374cdddc6b46541cd51739f93a424142d9577a00104a8c9906df not found: ID does not exist" Jul 29 13:14:34.605859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.605821 2563 scope.go:117] "RemoveContainer" containerID="00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6" Jul 29 13:14:34.606071 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:14:34.606055 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6\": container with ID starting with 00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6 not found: ID does not exist" containerID="00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6" Jul 29 13:14:34.606123 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.606076 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6"} err="failed to get container status \"00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6\": rpc error: code = NotFound desc = could not find container \"00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6\": container with ID starting with 00124a71f44da5d9b4aabcdb3aae4a2abead967e8f4a8e23267bb55f2ff50be6 not found: ID does not exist" Jul 29 13:14:34.613010 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.612992 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph"] Jul 29 13:14:34.617443 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.617415 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-2f4f5e-openshift-default-d44449fbb-jtwph"] Jul 29 13:14:34.671353 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671323 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzdwf\" (UniqueName: \"kubernetes.io/projected/1b48ec5f-13e8-4734-9546-614d7db6ea27-kube-api-access-nzdwf\") pod \"payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671488 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b48ec5f-13e8-4734-9546-614d7db6ea27-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671554 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqjwz\" (UniqueName: \"kubernetes.io/projected/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-kube-api-access-tqjwz\") pod \"payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671709 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671722 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671731 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671739 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671748 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671756 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.671816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.671765 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsgj8\" (UniqueName: \"kubernetes.io/projected/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c-kube-api-access-nsgj8\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:14:34.772891 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.772850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.773045 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.772923 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzdwf\" (UniqueName: \"kubernetes.io/projected/1b48ec5f-13e8-4734-9546-614d7db6ea27-kube-api-access-nzdwf\") pod \"payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.773162 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.773054 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b48ec5f-13e8-4734-9546-614d7db6ea27-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.773162 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.773127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjwz\" (UniqueName: \"kubernetes.io/projected/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-kube-api-access-tqjwz\") pod \"payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.773951 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.773908 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.774161 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.773988 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b48ec5f-13e8-4734-9546-614d7db6ea27-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.782080 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.782021 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjwz\" (UniqueName: \"kubernetes.io/projected/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-kube-api-access-tqjwz\") pod \"payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.782080 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.782032 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzdwf\" (UniqueName: \"kubernetes.io/projected/1b48ec5f-13e8-4734-9546-614d7db6ea27-kube-api-access-nzdwf\") pod \"payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.841207 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.841180 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:34.902314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.902069 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:34.972493 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:34.972396 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn"] Jul 29 13:14:34.973746 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:14:34.973691 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b48ec5f_13e8_4734_9546_614d7db6ea27.slice/crio-36dcef898ea2fb2453d6c1dca8b24ba34a63e2fe314105dfc4de4513559bcdab WatchSource:0}: Error finding container 36dcef898ea2fb2453d6c1dca8b24ba34a63e2fe314105dfc4de4513559bcdab: Status 404 returned error can't find the container with id 36dcef898ea2fb2453d6c1dca8b24ba34a63e2fe314105dfc4de4513559bcdab Jul 29 13:14:35.042618 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.042597 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k"] Jul 29 13:14:35.045037 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:14:35.045012 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e87cebb_139f_4782_b5eb_bbdfe49ebc1b.slice/crio-1b69123f0846a5c5dfb237d76546f187e242750b6cbd69caae7316082f874a2c WatchSource:0}: Error finding container 1b69123f0846a5c5dfb237d76546f187e242750b6cbd69caae7316082f874a2c: Status 404 returned error can't find the container with id 1b69123f0846a5c5dfb237d76546f187e242750b6cbd69caae7316082f874a2c Jul 29 13:14:35.586475 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.586429 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" event={"ID":"1b48ec5f-13e8-4734-9546-614d7db6ea27","Type":"ContainerStarted","Data":"f914a10f8aa86622cade1b9e6215824be1a55d4998087a8b4ed2415848ad072f"} Jul 29 13:14:35.586645 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.586513 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" event={"ID":"1b48ec5f-13e8-4734-9546-614d7db6ea27","Type":"ContainerStarted","Data":"36dcef898ea2fb2453d6c1dca8b24ba34a63e2fe314105dfc4de4513559bcdab"} Jul 29 13:14:35.586645 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.586621 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:35.588028 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.588005 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" event={"ID":"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b","Type":"ContainerStarted","Data":"1715973ad420a56bdaa70f2e37fd0059ca89b37ed45ed1cdc984a2cb65f909d5"} Jul 29 13:14:35.588133 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.588044 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" event={"ID":"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b","Type":"ContainerStarted","Data":"1b69123f0846a5c5dfb237d76546f187e242750b6cbd69caae7316082f874a2c"} Jul 29 13:14:35.588133 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.588078 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:35.604986 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.604944 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" podStartSLOduration=1.6049317090000002 podStartE2EDuration="1.604931709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:14:35.603666918 +0000 UTC m=+2948.506863591" watchObservedRunningTime="2026-07-29 13:14:35.604931709 +0000 UTC m=+2948.508128381" Jul 29 13:14:35.621609 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.621552 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" podStartSLOduration=1.621535078 podStartE2EDuration="1.621535078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:14:35.619182536 +0000 UTC m=+2948.522379214" watchObservedRunningTime="2026-07-29 13:14:35.621535078 +0000 UTC m=+2948.524731758" Jul 29 13:14:35.658688 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:35.658660 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a3b8eb6-a9d5-49a1-ba21-50f13e70315c" path="/var/lib/kubelet/pods/4a3b8eb6-a9d5-49a1-ba21-50f13e70315c/volumes" Jul 29 13:14:45.637446 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:45.637414 2563 generic.go:356] "Generic (PLEG): container finished" podID="936be956-5115-4a68-be8e-95b6c378a023" containerID="c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888" exitCode=0 Jul 29 13:14:45.637869 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:45.637490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" event={"ID":"936be956-5115-4a68-be8e-95b6c378a023","Type":"ContainerDied","Data":"c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888"} Jul 29 13:14:45.637869 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:45.637859 2563 scope.go:117] "RemoveContainer" containerID="c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888" Jul 29 13:14:45.638889 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:45.638864 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:45.638940 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:45.638927 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:45.638974 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:45.638961 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:46.594263 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:46.594219 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" podUID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.65:9004: connect: connection refused" Jul 29 13:14:46.594618 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:46.594598 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:14:46.646862 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:46.645887 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" event={"ID":"936be956-5115-4a68-be8e-95b6c378a023","Type":"ContainerStarted","Data":"aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae"} Jul 29 13:14:47.483813 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:47.483775 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:47.485339 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:47.485309 2563 patch_prober.go:28] interesting pod/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 29 13:14:47.485498 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:47.485367 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 29 13:14:47.565796 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:47.565770 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:48.484208 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:48.484159 2563 patch_prober.go:28] interesting pod/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 29 13:14:48.484719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:48.484233 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 29 13:14:49.484920 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:49.484833 2563 patch_prober.go:28] interesting pod/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 29 13:14:49.485482 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:49.485431 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 29 13:14:50.484000 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:50.483961 2563 patch_prober.go:28] interesting pod/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 29 13:14:50.484264 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:50.484231 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 29 13:14:51.487975 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:51.487947 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:51.488934 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:51.488913 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:14:56.593722 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:56.593692 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:14:58.622519 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:58.622482 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:58.989594 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:58.989544 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:59.289866 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.289792 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx"] Jul 29 13:14:59.292228 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.292211 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.295315 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.295296 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-11a904-openshift-default-dockercfg-9nkhc\"" Jul 29 13:14:59.309404 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.309380 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx"] Jul 29 13:14:59.333890 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.333864 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:59.380837 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.380805 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cd68cc-f687-4be9-8302-7834edf3f935-istiod-ca-cert\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.380956 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.380857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-istio-token\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.380956 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.380888 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cd68cc-f687-4be9-8302-7834edf3f935-istio-podinfo\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.380956 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.380932 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-socket\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.381152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.381027 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-credential-socket\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.381152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.381061 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-certs\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.381152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.381109 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-envoy\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.381152 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.381145 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-data\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.381359 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.381222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg4q5\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-kube-api-access-bg4q5\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.481942 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.481913 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-credential-socket\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.481942 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.481943 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-certs\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482154 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.481968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-envoy\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482154 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.481998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-data\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482154 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482031 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg4q5\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-kube-api-access-bg4q5\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482154 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cd68cc-f687-4be9-8302-7834edf3f935-istiod-ca-cert\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482154 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482089 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-istio-token\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482154 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cd68cc-f687-4be9-8302-7834edf3f935-istio-podinfo\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482165 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-socket\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482357 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-credential-socket\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482435 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-certs\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482913 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482884 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-socket\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.482913 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.482903 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-data\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.483094 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.483059 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cd68cc-f687-4be9-8302-7834edf3f935-istiod-ca-cert\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.484300 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.484278 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-envoy\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.484422 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.484390 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cd68cc-f687-4be9-8302-7834edf3f935-istio-podinfo\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.492083 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.492064 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg4q5\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-kube-api-access-bg4q5\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.492744 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.492723 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-istio-token\") pod \"e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.556310 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.556255 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:14:59.602427 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.602405 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:14:59.734521 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.734483 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx"] Jul 29 13:14:59.736692 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:14:59.736669 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40cd68cc_f687_4be9_8302_7834edf3f935.slice/crio-d2c2cc7d123cc19e8f557effb03e386d2e731b5eca4afbde3e9150c553e9426a WatchSource:0}: Error finding container d2c2cc7d123cc19e8f557effb03e386d2e731b5eca4afbde3e9150c553e9426a: Status 404 returned error can't find the container with id d2c2cc7d123cc19e8f557effb03e386d2e731b5eca4afbde3e9150c553e9426a Jul 29 13:14:59.738696 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.738667 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:59.738789 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.738743 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:14:59.738850 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:14:59.738808 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:15:00.708789 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:00.708755 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" event={"ID":"40cd68cc-f687-4be9-8302-7834edf3f935","Type":"ContainerStarted","Data":"2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00"} Jul 29 13:15:00.708991 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:00.708813 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" event={"ID":"40cd68cc-f687-4be9-8302-7834edf3f935","Type":"ContainerStarted","Data":"d2c2cc7d123cc19e8f557effb03e386d2e731b5eca4afbde3e9150c553e9426a"} Jul 29 13:15:00.737859 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:00.737810 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podStartSLOduration=1.737796151 podStartE2EDuration="1.737796151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:14:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:15:00.734937266 +0000 UTC m=+2973.638133940" watchObservedRunningTime="2026-07-29 13:15:00.737796151 +0000 UTC m=+2973.640992827" Jul 29 13:15:01.602676 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:01.602641 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:01.607236 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:01.607212 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:01.713035 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:01.713010 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:01.713846 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:01.713829 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:02.779175 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:02.779138 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:02.861416 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:02.861374 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:02.930118 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:02.930087 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:03.064930 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.064839 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:03.336705 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.336617 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:03.439913 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.439879 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:03.450784 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.450760 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm"] Jul 29 13:15:03.453185 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.453169 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.455585 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.455564 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-11a904\"" Jul 29 13:15:03.464789 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.464769 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm"] Jul 29 13:15:03.518972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.518948 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9l2x\" (UniqueName: \"kubernetes.io/projected/d1b9a761-7df1-44db-818a-38bae518d3bb-kube-api-access-z9l2x\") pod \"payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.519086 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.518999 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1b9a761-7df1-44db-818a-38bae518d3bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.620398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.620313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1b9a761-7df1-44db-818a-38bae518d3bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.620566 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.620425 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9l2x\" (UniqueName: \"kubernetes.io/projected/d1b9a761-7df1-44db-818a-38bae518d3bb-kube-api-access-z9l2x\") pod \"payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.621027 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.621004 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1b9a761-7df1-44db-818a-38bae518d3bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.628181 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.628156 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9l2x\" (UniqueName: \"kubernetes.io/projected/d1b9a761-7df1-44db-818a-38bae518d3bb-kube-api-access-z9l2x\") pod \"payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.764575 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.764545 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:03.895254 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:03.895185 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm"] Jul 29 13:15:03.897275 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:15:03.897249 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b9a761_7df1_44db_818a_38bae518d3bb.slice/crio-3d518137079d1fbfa51aefc6c2eb01230ed8cde760c1ddd93a3f47167180f676 WatchSource:0}: Error finding container 3d518137079d1fbfa51aefc6c2eb01230ed8cde760c1ddd93a3f47167180f676: Status 404 returned error can't find the container with id 3d518137079d1fbfa51aefc6c2eb01230ed8cde760c1ddd93a3f47167180f676 Jul 29 13:15:04.729160 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:04.729113 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" event={"ID":"d1b9a761-7df1-44db-818a-38bae518d3bb","Type":"ContainerStarted","Data":"e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5"} Jul 29 13:15:04.729386 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:04.729168 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" event={"ID":"d1b9a761-7df1-44db-818a-38bae518d3bb","Type":"ContainerStarted","Data":"3d518137079d1fbfa51aefc6c2eb01230ed8cde760c1ddd93a3f47167180f676"} Jul 29 13:15:04.729386 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:04.729221 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:04.745512 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:04.745453 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" podStartSLOduration=1.7454394629999999 podStartE2EDuration="1.745439463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:15:04.744743838 +0000 UTC m=+2977.647940516" watchObservedRunningTime="2026-07-29 13:15:04.745439463 +0000 UTC m=+2977.648636137" Jul 29 13:15:15.542196 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:15.542152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:15.596322 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:15.596277 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:15.735648 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:15.735611 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:15:17.159853 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.159817 2563 patch_prober.go:28] interesting pod/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": read tcp 10.134.0.2:50350->10.134.0.66:15021: read: connection reset by peer" start-of-body= Jul 29 13:15:17.160367 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.159909 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": read tcp 10.134.0.2:50350->10.134.0.66:15021: read: connection reset by peer" Jul 29 13:15:17.790104 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.790075 2563 generic.go:356] "Generic (PLEG): container finished" podID="40cd68cc-f687-4be9-8302-7834edf3f935" containerID="2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00" exitCode=0 Jul 29 13:15:17.790284 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.790150 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" event={"ID":"40cd68cc-f687-4be9-8302-7834edf3f935","Type":"ContainerDied","Data":"2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00"} Jul 29 13:15:17.790557 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.790542 2563 scope.go:117] "RemoveContainer" containerID="2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00" Jul 29 13:15:17.791541 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.791518 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:15:17.791605 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.791580 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:15:17.791640 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:17.791612 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:15:18.795972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:18.795942 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" event={"ID":"40cd68cc-f687-4be9-8302-7834edf3f935","Type":"ContainerStarted","Data":"d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37"} Jul 29 13:15:19.602872 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:19.602829 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:19.603118 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:19.603096 2563 patch_prober.go:28] interesting pod/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 29 13:15:19.603175 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:19.603151 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 29 13:15:20.602929 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:20.602891 2563 patch_prober.go:28] interesting pod/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 29 13:15:20.603332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:20.602949 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 29 13:15:21.602804 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:21.602761 2563 patch_prober.go:28] interesting pod/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 29 13:15:21.602974 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:21.602824 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 29 13:15:22.602972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:22.602935 2563 patch_prober.go:28] interesting pod/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body= Jul 29 13:15:22.603362 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:22.602991 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" Jul 29 13:15:23.606972 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:23.606936 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:23.607500 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:23.607351 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:23.608021 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:23.608000 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:15:27.726471 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:27.726420 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:30.580708 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:30.580673 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:15:54.475809 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:15:54.475769 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:16:35.718919 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:16:35.718811 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:17:32.818172 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:17:32.818133 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:17:32.922755 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:17:32.922721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:18:48.249731 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:18:48.249686 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:18:57.119797 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:18:57.119758 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:19:22.112800 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:19:22.112722 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:20:48.020031 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:20:48.019999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:20:48.083220 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:20:48.083189 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:20:50.796317 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:20:50.796235 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:21:13.373943 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:21:13.373901 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:21:23.585396 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:21:23.585362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:21:23.681685 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:21:23.681657 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:22:48.119040 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:22:48.118946 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:22:49.345425 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:22:49.345378 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:22:58.822838 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:22:58.822790 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:23:22.188934 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:23:22.188899 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:24:50.729315 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:24:50.729281 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:24:51.499002 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:24:51.498970 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:24:55.130083 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:24:55.130048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:11.492385 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.492347 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:11.782256 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.782180 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf"] Jul 29 13:25:11.785504 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.785488 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.787981 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.787958 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-86053a-openshift-default-dockercfg-dmmrz\"" Jul 29 13:25:11.797358 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.797334 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf"] Jul 29 13:25:11.805520 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.805494 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:11.871164 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871054 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-certs\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871164 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871125 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-token\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871164 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-data\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871476 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-socket\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871476 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871231 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-credential-socket\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871476 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istiod-ca-cert\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871476 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-podinfo\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871476 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871326 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f55hd\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-kube-api-access-f55hd\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.871762 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.871495 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-envoy\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.971969 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.971929 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-token\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.971969 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.971972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-data\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.971990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-socket\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-credential-socket\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972033 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istiod-ca-cert\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972058 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-podinfo\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f55hd\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-kube-api-access-f55hd\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972131 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-envoy\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972241 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972169 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-certs\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972672 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972407 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-data\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972672 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972450 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-socket\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972672 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972557 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-certs\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.972672 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.972574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-credential-socket\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.973037 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.973017 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istiod-ca-cert\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.974559 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.974531 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-envoy\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.974706 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.974677 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-podinfo\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.981112 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.981087 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f55hd\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-kube-api-access-f55hd\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:11.981673 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:11.981652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-token\") pod \"e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:12.096030 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.095965 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:12.228930 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.228894 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:12.236328 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.236303 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf"] Jul 29 13:25:12.239077 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:25:12.239053 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad6115e_9e42_49df_a8ba_8db3c92d566a.slice/crio-917a34a36aa30934075293a287f2c5ab944e0e94372a58b77719c3643c9dff97 WatchSource:0}: Error finding container 917a34a36aa30934075293a287f2c5ab944e0e94372a58b77719c3643c9dff97: Status 404 returned error can't find the container with id 917a34a36aa30934075293a287f2c5ab944e0e94372a58b77719c3643c9dff97 Jul 29 13:25:12.241398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.241369 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:25:12.241488 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.241428 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:25:12.241488 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.241455 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:25:12.529335 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.529303 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" event={"ID":"1ad6115e-9e42-49df-a8ba-8db3c92d566a","Type":"ContainerStarted","Data":"b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f"} Jul 29 13:25:12.529729 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.529362 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" event={"ID":"1ad6115e-9e42-49df-a8ba-8db3c92d566a","Type":"ContainerStarted","Data":"917a34a36aa30934075293a287f2c5ab944e0e94372a58b77719c3643c9dff97"} Jul 29 13:25:12.551899 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:12.551849 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podStartSLOduration=1.5518346570000001 podStartE2EDuration="1.551834657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:25:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:25:12.549886629 +0000 UTC m=+3585.453083330" watchObservedRunningTime="2026-07-29 13:25:12.551834657 +0000 UTC m=+3585.455031334" Jul 29 13:25:13.096364 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:13.096327 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:13.101148 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:13.101122 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:13.534412 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:13.534378 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:13.535346 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:13.535328 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:14.494029 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:14.493994 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:14.590735 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:14.590698 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:14.688628 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:14.688596 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:14.790852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:14.790763 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:15.186514 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:15.186425 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:15.299982 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:15.299950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:29.576347 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:29.575596 2563 patch_prober.go:28] interesting pod/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 29 13:25:29.576347 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:29.575671 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 29 13:25:30.620398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:30.620363 2563 generic.go:356] "Generic (PLEG): container finished" podID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerID="b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f" exitCode=0 Jul 29 13:25:30.620794 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:30.620426 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" event={"ID":"1ad6115e-9e42-49df-a8ba-8db3c92d566a","Type":"ContainerDied","Data":"b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f"} Jul 29 13:25:30.620838 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:30.620810 2563 scope.go:117] "RemoveContainer" containerID="b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f" Jul 29 13:25:30.621822 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:30.621796 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:25:30.621890 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:30.621880 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:25:30.621930 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:30.621915 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:25:31.627591 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:31.627557 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" event={"ID":"1ad6115e-9e42-49df-a8ba-8db3c92d566a","Type":"ContainerStarted","Data":"cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead"} Jul 29 13:25:32.096786 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:32.096742 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:32.097040 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:32.097016 2563 patch_prober.go:28] interesting pod/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 29 13:25:32.097102 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:32.097082 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 29 13:25:33.096949 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:33.096911 2563 patch_prober.go:28] interesting pod/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 29 13:25:33.097350 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:33.096967 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 29 13:25:34.096407 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:34.096370 2563 patch_prober.go:28] interesting pod/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 29 13:25:34.096596 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:34.096428 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 29 13:25:35.096429 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:35.096389 2563 patch_prober.go:28] interesting pod/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Jul 29 13:25:35.096834 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:35.096452 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Jul 29 13:25:36.100408 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:36.100379 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:36.100804 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:36.100788 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:36.101583 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:36.101565 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:25:39.519529 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:39.519478 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:40.228844 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:40.228802 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:40.562708 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:40.562629 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:41.117326 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:41.117291 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:47.446684 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:47.446633 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:47.703151 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:47.703058 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:47.845804 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:47.845764 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:47.960047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:47.960014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:48.325326 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:48.325245 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:25:48.439380 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:25:48.439336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:26:12.248104 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:26:12.248054 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:26:21.781403 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:26:21.781359 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:26:59.471175 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:26:59.471130 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:06.931746 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:06.931714 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:07.239983 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.238479 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj"] Jul 29 13:27:07.244523 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.244497 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.247484 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.247439 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-08bb55-openshift-default-dockercfg-zchwb\"" Jul 29 13:27:07.253740 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.253718 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj"] Jul 29 13:27:07.266876 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.266848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:07.366236 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.366203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:07.394749 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.394709 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-certs\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.394943 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.394753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istiod-ca-cert\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.394943 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.394791 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-envoy\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.394943 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.394848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-token\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.395123 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.394975 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-socket\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.395123 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.395006 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-credential-socket\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.395123 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.395041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-data\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.395123 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.395064 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-podinfo\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.395283 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.395163 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vsqb\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-kube-api-access-4vsqb\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496124 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496039 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-token\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496124 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-socket\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496135 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-credential-socket\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496168 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-data\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496191 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-podinfo\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vsqb\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-kube-api-access-4vsqb\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496271 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-certs\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496332 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istiod-ca-cert\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496653 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496338 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-envoy\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496653 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496594 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-socket\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496653 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496636 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-data\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496809 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-credential-socket\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.496888 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.496867 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-certs\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.497054 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.497039 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istiod-ca-cert\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.498514 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.498492 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-envoy\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.498743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.498725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-podinfo\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.507590 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.507570 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-token\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.507832 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.507815 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vsqb\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-kube-api-access-4vsqb\") pod \"e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.555285 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.555263 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:07.685546 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.685372 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj"] Jul 29 13:27:07.688319 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:27:07.688292 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07112c82_a7cd_4eac_bf4b_5b99e76ab248.slice/crio-fa871de039173e14cfc9c460d21e73a2d65b2f66b24b813dbdefe024122931f4 WatchSource:0}: Error finding container fa871de039173e14cfc9c460d21e73a2d65b2f66b24b813dbdefe024122931f4: Status 404 returned error can't find the container with id fa871de039173e14cfc9c460d21e73a2d65b2f66b24b813dbdefe024122931f4 Jul 29 13:27:07.690323 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.690297 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:27:07.690403 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.690355 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:27:07.690403 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:07.690380 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:27:08.093675 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:08.093594 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" event={"ID":"07112c82-a7cd-4eac-bf4b-5b99e76ab248","Type":"ContainerStarted","Data":"19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b"} Jul 29 13:27:08.093675 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:08.093643 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" event={"ID":"07112c82-a7cd-4eac-bf4b-5b99e76ab248","Type":"ContainerStarted","Data":"fa871de039173e14cfc9c460d21e73a2d65b2f66b24b813dbdefe024122931f4"} Jul 29 13:27:08.116986 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:08.116941 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podStartSLOduration=1.116901957 podStartE2EDuration="1.116901957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 13:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:27:08.115973076 +0000 UTC m=+3701.019169756" watchObservedRunningTime="2026-07-29 13:27:08.116901957 +0000 UTC m=+3701.020098633" Jul 29 13:27:08.555590 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:08.555562 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:08.560361 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:08.560336 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:09.098592 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:09.098565 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:09.099539 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:09.099518 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:10.054632 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:10.054594 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:10.150470 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:10.150428 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:10.260881 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:10.260843 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:10.882594 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:10.882557 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:10.984914 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:10.984858 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:11.083207 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:11.083172 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:31.221506 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:31.221455 2563 generic.go:356] "Generic (PLEG): container finished" podID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerID="19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b" exitCode=0 Jul 29 13:27:31.221907 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:31.221526 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" event={"ID":"07112c82-a7cd-4eac-bf4b-5b99e76ab248","Type":"ContainerDied","Data":"19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b"} Jul 29 13:27:31.221952 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:31.221913 2563 scope.go:117] "RemoveContainer" containerID="19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b" Jul 29 13:27:31.222913 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:31.222886 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:27:31.222977 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:31.222963 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:27:31.223017 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:31.222999 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 13:27:32.228356 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:32.228322 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" event={"ID":"07112c82-a7cd-4eac-bf4b-5b99e76ab248","Type":"ContainerStarted","Data":"5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476"} Jul 29 13:27:32.556139 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:32.556050 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:32.556283 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:32.556197 2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 29 13:27:32.556283 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:32.556243 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 29 13:27:33.555826 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:33.555792 2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 29 13:27:33.556176 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:33.555846 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 29 13:27:34.556108 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:34.556070 2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 29 13:27:34.556534 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:34.556127 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 29 13:27:35.113035 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:35.112999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:35.556255 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:35.556224 2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Jul 29 13:27:35.556644 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:35.556276 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Jul 29 13:27:36.559354 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:36.559319 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:36.559749 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:36.559676 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:36.560231 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:36.560212 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:40.922528 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:40.922488 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:41.017268 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:41.017237 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:41.080862 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:41.080812 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:41.408903 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:41.408877 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:47.550574 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:47.550539 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:47.657292 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:47.657249 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:47.792973 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:47.792940 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj"] Jul 29 13:27:47.793183 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:47.793158 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" containerID="cri-o://5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476" gracePeriod=30 Jul 29 13:27:47.818056 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:47.817979 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:48.048738 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:48.048698 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:27:51.562091 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:51.562058 2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 29 13:27:51.562513 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:51.562113 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 13:27:53.039881 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.039856 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:53.094744 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.094672 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vsqb\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-kube-api-access-4vsqb\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.094744 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.094705 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-token\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.094744 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.094725 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istiod-ca-cert\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.094978 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.094859 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-podinfo\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.094978 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.094933 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-credential-socket\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.095085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.094979 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-certs\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.095085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095037 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-socket\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.095085 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095061 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-envoy\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.095248 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095085 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:27:53.095248 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095099 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-data\") pod \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\" (UID: \"07112c82-a7cd-4eac-bf4b-5b99e76ab248\") " Jul 29 13:27:53.095354 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095226 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:27:53.095411 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095373 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:27:53.095539 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095512 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:27:53.095614 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095581 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.095614 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095600 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.095719 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.095633 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.096211 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.096186 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-data" (OuterVolumeSpecName: "istio-data") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:27:53.097269 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.097246 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-token" (OuterVolumeSpecName: "istio-token") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:27:53.097358 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.097334 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:27:53.097398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.097354 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:27:53.097398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.097356 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-kube-api-access-4vsqb" (OuterVolumeSpecName: "kube-api-access-4vsqb") pod "07112c82-a7cd-4eac-bf4b-5b99e76ab248" (UID: "07112c82-a7cd-4eac-bf4b-5b99e76ab248"). InnerVolumeSpecName "kube-api-access-4vsqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:27:53.197024 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.197001 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.197024 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.197023 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.197128 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.197033 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.197128 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.197040 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.197128 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.197048 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vsqb\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-kube-api-access-4vsqb\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.197128 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.197057 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07112c82-a7cd-4eac-bf4b-5b99e76ab248-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:27:53.329105 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.329079 2563 generic.go:356] "Generic (PLEG): container finished" podID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerID="5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476" exitCode=0 Jul 29 13:27:53.329196 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.329126 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" event={"ID":"07112c82-a7cd-4eac-bf4b-5b99e76ab248","Type":"ContainerDied","Data":"5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476"} Jul 29 13:27:53.329196 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.329131 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" Jul 29 13:27:53.329196 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.329168 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj" event={"ID":"07112c82-a7cd-4eac-bf4b-5b99e76ab248","Type":"ContainerDied","Data":"fa871de039173e14cfc9c460d21e73a2d65b2f66b24b813dbdefe024122931f4"} Jul 29 13:27:53.329196 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.329190 2563 scope.go:117] "RemoveContainer" containerID="5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476" Jul 29 13:27:53.337999 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.337981 2563 scope.go:117] "RemoveContainer" containerID="19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b" Jul 29 13:27:53.345405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.345388 2563 scope.go:117] "RemoveContainer" containerID="5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476" Jul 29 13:27:53.345695 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:27:53.345676 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476\": container with ID starting with 5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476 not found: ID does not exist" containerID="5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476" Jul 29 13:27:53.345767 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.345701 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476"} err="failed to get container status \"5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476\": rpc error: code = NotFound desc = could not find container \"5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476\": container with ID starting with 5a04108a2adc3d76b0e86bdea5a9cfb26e3d53434109d1c6bd60076af169d476 not found: ID does not exist" Jul 29 13:27:53.345767 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.345721 2563 scope.go:117] "RemoveContainer" containerID="19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b" Jul 29 13:27:53.345950 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:27:53.345933 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b\": container with ID starting with 19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b not found: ID does not exist" containerID="19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b" Jul 29 13:27:53.345993 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.345954 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b"} err="failed to get container status \"19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b\": rpc error: code = NotFound desc = could not find container \"19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b\": container with ID starting with 19693ec8e857292b9cbce19c5a015fcb5022c524082aeeff8a9df6f1f371f55b not found: ID does not exist" Jul 29 13:27:53.351156 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.351135 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj"] Jul 29 13:27:53.355909 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.355888 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-08bb55-openshift-default-555df98c8d-jw8bj"] Jul 29 13:27:53.658775 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:27:53.658698 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" path="/var/lib/kubelet/pods/07112c82-a7cd-4eac-bf4b-5b99e76ab248/volumes" Jul 29 13:28:07.181263 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:07.181230 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:08.430520 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:08.430482 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:12.763450 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:12.763411 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:31.233723 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:31.233689 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:49.245414 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:49.245380 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:49.961662 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:49.961627 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:55.553481 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:55.553424 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:55.654204 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:55.654170 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:55.831740 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:55.831667 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf"] Jul 29 13:28:55.831890 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:55.831850 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" containerID="cri-o://cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead" gracePeriod=30 Jul 29 13:28:55.857801 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:55.857775 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:28:56.095921 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:28:56.095834 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:01.082599 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.082577 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:29:01.133521 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133494 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-socket\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133657 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133535 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-data\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133657 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133556 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-envoy\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133657 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133608 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f55hd\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-kube-api-access-f55hd\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133657 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133634 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-certs\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133663 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-podinfo\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133707 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-credential-socket\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133755 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-token\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.133852 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133784 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istiod-ca-cert\") pod \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\" (UID: \"1ad6115e-9e42-49df-a8ba-8db3c92d566a\") " Jul 29 13:29:01.134027 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.133749 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:01.134027 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.134002 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:01.134130 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.134076 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.134293 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.134271 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:01.134376 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.134321 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:29:01.134454 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.134423 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:01.135819 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.135793 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-kube-api-access-f55hd" (OuterVolumeSpecName: "kube-api-access-f55hd") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "kube-api-access-f55hd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:01.135917 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.135876 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:29:01.136119 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.136095 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:01.136178 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.136162 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ad6115e-9e42-49df-a8ba-8db3c92d566a" (UID: "1ad6115e-9e42-49df-a8ba-8db3c92d566a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:01.234811 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234780 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.234811 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234811 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.234811 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234820 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.235012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234828 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.235012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234836 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f55hd\" (UniqueName: \"kubernetes.io/projected/1ad6115e-9e42-49df-a8ba-8db3c92d566a-kube-api-access-f55hd\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.235012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234843 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.235012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234851 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ad6115e-9e42-49df-a8ba-8db3c92d566a-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.235012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.234861 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ad6115e-9e42-49df-a8ba-8db3c92d566a-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:01.653282 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.653208 2563 generic.go:356] "Generic (PLEG): container finished" podID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerID="cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead" exitCode=0 Jul 29 13:29:01.653448 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.653310 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" Jul 29 13:29:01.658029 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.658000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" event={"ID":"1ad6115e-9e42-49df-a8ba-8db3c92d566a","Type":"ContainerDied","Data":"cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead"} Jul 29 13:29:01.658141 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.658048 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf" event={"ID":"1ad6115e-9e42-49df-a8ba-8db3c92d566a","Type":"ContainerDied","Data":"917a34a36aa30934075293a287f2c5ab944e0e94372a58b77719c3643c9dff97"} Jul 29 13:29:01.658141 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.658081 2563 scope.go:117] "RemoveContainer" containerID="cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead" Jul 29 13:29:01.668347 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.668328 2563 scope.go:117] "RemoveContainer" containerID="b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f" Jul 29 13:29:01.675993 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.675972 2563 scope.go:117] "RemoveContainer" containerID="cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead" Jul 29 13:29:01.676247 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:01.676229 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead\": container with ID starting with cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead not found: ID does not exist" containerID="cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead" Jul 29 13:29:01.676307 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.676253 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead"} err="failed to get container status \"cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead\": rpc error: code = NotFound desc = could not find container \"cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead\": container with ID starting with cc2a538a8f92b1017a7454223fdbeaac27ad9c4132747207f1cbb984546a8ead not found: ID does not exist" Jul 29 13:29:01.676307 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.676270 2563 scope.go:117] "RemoveContainer" containerID="b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f" Jul 29 13:29:01.676536 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:01.676515 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f\": container with ID starting with b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f not found: ID does not exist" containerID="b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f" Jul 29 13:29:01.676583 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.676542 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f"} err="failed to get container status \"b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f\": rpc error: code = NotFound desc = could not find container \"b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f\": container with ID starting with b8c4adf407097f429afaa3c511f94bfbfd88564b95b9a3344986f7ac25dd173f not found: ID does not exist" Jul 29 13:29:01.680285 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.680264 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf"] Jul 29 13:29:01.684916 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:01.684895 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-86053a-openshift-default-648d5b7476-t2fnf"] Jul 29 13:29:03.658278 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:03.658240 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" path="/var/lib/kubelet/pods/1ad6115e-9e42-49df-a8ba-8db3c92d566a/volumes" Jul 29 13:29:06.664447 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:06.664411 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:06.727688 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:06.727656 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:06.784810 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:06.784779 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:06.847649 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:06.847619 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:07.228887 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:07.228848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:13.498884 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:13.498848 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:13.621611 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:13.621578 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:13.810073 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:13.809996 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:14.030222 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:14.030159 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:24.616179 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.616141 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn"] Jul 29 13:29:24.616653 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.616393 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" podUID="1b48ec5f-13e8-4734-9546-614d7db6ea27" containerName="payload-processing" containerID="cri-o://f914a10f8aa86622cade1b9e6215824be1a55d4998087a8b4ed2415848ad072f" gracePeriod=30 Jul 29 13:29:24.622174 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.622151 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k"] Jul 29 13:29:24.622546 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.622499 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" podUID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" containerName="payload-pre-processing" containerID="cri-o://1715973ad420a56bdaa70f2e37fd0059ca89b37ed45ed1cdc984a2cb65f909d5" gracePeriod=30 Jul 29 13:29:24.645244 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.645217 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:24.747250 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.747221 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:24.770218 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.770183 2563 generic.go:356] "Generic (PLEG): container finished" podID="1b48ec5f-13e8-4734-9546-614d7db6ea27" containerID="f914a10f8aa86622cade1b9e6215824be1a55d4998087a8b4ed2415848ad072f" exitCode=0 Jul 29 13:29:24.773314 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.771296 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" event={"ID":"1b48ec5f-13e8-4734-9546-614d7db6ea27","Type":"ContainerDied","Data":"f914a10f8aa86622cade1b9e6215824be1a55d4998087a8b4ed2415848ad072f"} Jul 29 13:29:24.779765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.779742 2563 generic.go:356] "Generic (PLEG): container finished" podID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" containerID="1715973ad420a56bdaa70f2e37fd0059ca89b37ed45ed1cdc984a2cb65f909d5" exitCode=0 Jul 29 13:29:24.780063 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.779820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" event={"ID":"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b","Type":"ContainerDied","Data":"1715973ad420a56bdaa70f2e37fd0059ca89b37ed45ed1cdc984a2cb65f909d5"} Jul 29 13:29:24.821228 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.821110 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:24.892150 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.892130 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:29:24.907974 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:24.907957 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:29:25.032122 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032085 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-plugins-config-volume\") pod \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " Jul 29 13:29:25.032122 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032132 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b48ec5f-13e8-4734-9546-614d7db6ea27-plugins-config-volume\") pod \"1b48ec5f-13e8-4734-9546-614d7db6ea27\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " Jul 29 13:29:25.032328 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032156 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzdwf\" (UniqueName: \"kubernetes.io/projected/1b48ec5f-13e8-4734-9546-614d7db6ea27-kube-api-access-nzdwf\") pod \"1b48ec5f-13e8-4734-9546-614d7db6ea27\" (UID: \"1b48ec5f-13e8-4734-9546-614d7db6ea27\") " Jul 29 13:29:25.032398 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032332 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqjwz\" (UniqueName: \"kubernetes.io/projected/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-kube-api-access-tqjwz\") pod \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\" (UID: \"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b\") " Jul 29 13:29:25.032485 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032454 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" (UID: "4e87cebb-139f-4782-b5eb-bbdfe49ebc1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:29:25.032551 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032522 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b48ec5f-13e8-4734-9546-614d7db6ea27-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b48ec5f-13e8-4734-9546-614d7db6ea27" (UID: "1b48ec5f-13e8-4734-9546-614d7db6ea27"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:29:25.032720 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032701 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:25.032779 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.032725 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b48ec5f-13e8-4734-9546-614d7db6ea27-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:25.034063 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.034042 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b48ec5f-13e8-4734-9546-614d7db6ea27-kube-api-access-nzdwf" (OuterVolumeSpecName: "kube-api-access-nzdwf") pod "1b48ec5f-13e8-4734-9546-614d7db6ea27" (UID: "1b48ec5f-13e8-4734-9546-614d7db6ea27"). InnerVolumeSpecName "kube-api-access-nzdwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:25.034243 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.034224 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-kube-api-access-tqjwz" (OuterVolumeSpecName: "kube-api-access-tqjwz") pod "4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" (UID: "4e87cebb-139f-4782-b5eb-bbdfe49ebc1b"). InnerVolumeSpecName "kube-api-access-tqjwz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:25.133740 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.133686 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzdwf\" (UniqueName: \"kubernetes.io/projected/1b48ec5f-13e8-4734-9546-614d7db6ea27-kube-api-access-nzdwf\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:25.133740 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.133706 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqjwz\" (UniqueName: \"kubernetes.io/projected/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b-kube-api-access-tqjwz\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:25.169078 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.169048 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:25.785738 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.785709 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" Jul 29 13:29:25.786113 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.785707 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn" event={"ID":"1b48ec5f-13e8-4734-9546-614d7db6ea27","Type":"ContainerDied","Data":"36dcef898ea2fb2453d6c1dca8b24ba34a63e2fe314105dfc4de4513559bcdab"} Jul 29 13:29:25.786113 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.785841 2563 scope.go:117] "RemoveContainer" containerID="f914a10f8aa86622cade1b9e6215824be1a55d4998087a8b4ed2415848ad072f" Jul 29 13:29:25.787359 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.787163 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" event={"ID":"4e87cebb-139f-4782-b5eb-bbdfe49ebc1b","Type":"ContainerDied","Data":"1b69123f0846a5c5dfb237d76546f187e242750b6cbd69caae7316082f874a2c"} Jul 29 13:29:25.787359 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.787181 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k" Jul 29 13:29:25.794542 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.794512 2563 scope.go:117] "RemoveContainer" containerID="1715973ad420a56bdaa70f2e37fd0059ca89b37ed45ed1cdc984a2cb65f909d5" Jul 29 13:29:25.804274 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.804251 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn"] Jul 29 13:29:25.806291 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.806274 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-f0ad55-5f496cbc78-vknkn"] Jul 29 13:29:25.815562 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.815537 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k"] Jul 29 13:29:25.817793 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:25.817773 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-f0ad55-95b5bbf6-5zb6k"] Jul 29 13:29:27.658420 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:27.658384 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b48ec5f-13e8-4734-9546-614d7db6ea27" path="/var/lib/kubelet/pods/1b48ec5f-13e8-4734-9546-614d7db6ea27/volumes" Jul 29 13:29:27.658774 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:27.658752 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" path="/var/lib/kubelet/pods/4e87cebb-139f-4782-b5eb-bbdfe49ebc1b/volumes" Jul 29 13:29:31.114251 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:31.114206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:31.176231 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:31.176204 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:31.369393 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:31.369306 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82"] Jul 29 13:29:31.369725 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:31.369664 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" containerID="cri-o://aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae" gracePeriod=30 Jul 29 13:29:31.446757 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:31.446721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:31.614558 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:31.614514 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:36.490049 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.490022 2563 patch_prober.go:28] interesting pod/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 29 13:29:36.490327 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.490071 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 29 13:29:36.614703 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.614681 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:29:36.728473 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728430 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-envoy\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728619 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728489 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-certs\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728619 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728522 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/936be956-5115-4a68-be8e-95b6c378a023-istio-podinfo\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728619 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728556 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-socket\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728619 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728575 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/936be956-5115-4a68-be8e-95b6c378a023-istiod-ca-cert\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728621 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r24ps\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-kube-api-access-r24ps\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728652 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-istio-token\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728684 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-data\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728710 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-credential-socket\") pod \"936be956-5115-4a68-be8e-95b6c378a023\" (UID: \"936be956-5115-4a68-be8e-95b6c378a023\") " Jul 29 13:29:36.728816 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728757 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:36.729083 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.728851 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:36.729083 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.729070 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:36.729250 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.729095 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.729250 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.729098 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/936be956-5115-4a68-be8e-95b6c378a023-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:29:36.729250 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.729114 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.729848 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.729815 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-data" (OuterVolumeSpecName: "istio-data") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:36.730757 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.730728 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:36.730877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.730810 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/936be956-5115-4a68-be8e-95b6c378a023-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:29:36.730923 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.730905 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-istio-token" (OuterVolumeSpecName: "istio-token") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:36.730965 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.730935 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-kube-api-access-r24ps" (OuterVolumeSpecName: "kube-api-access-r24ps") pod "936be956-5115-4a68-be8e-95b6c378a023" (UID: "936be956-5115-4a68-be8e-95b6c378a023"). InnerVolumeSpecName "kube-api-access-r24ps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:36.829812 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829787 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.829812 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829808 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/936be956-5115-4a68-be8e-95b6c378a023-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.829812 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829817 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/936be956-5115-4a68-be8e-95b6c378a023-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.829992 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829826 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r24ps\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-kube-api-access-r24ps\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.829992 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829834 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/936be956-5115-4a68-be8e-95b6c378a023-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.829992 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829843 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.829992 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.829851 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/936be956-5115-4a68-be8e-95b6c378a023-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:36.838140 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.838116 2563 generic.go:356] "Generic (PLEG): container finished" podID="936be956-5115-4a68-be8e-95b6c378a023" containerID="aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae" exitCode=0 Jul 29 13:29:36.838240 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.838165 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" Jul 29 13:29:36.838240 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.838201 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" event={"ID":"936be956-5115-4a68-be8e-95b6c378a023","Type":"ContainerDied","Data":"aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae"} Jul 29 13:29:36.838319 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.838255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82" event={"ID":"936be956-5115-4a68-be8e-95b6c378a023","Type":"ContainerDied","Data":"dd0e3d733468aff94195ff3198393e36a91c2f196fb1957fee4096c9363b8edf"} Jul 29 13:29:36.838319 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.838283 2563 scope.go:117] "RemoveContainer" containerID="aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae" Jul 29 13:29:36.847005 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.846990 2563 scope.go:117] "RemoveContainer" containerID="c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888" Jul 29 13:29:36.854192 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.854172 2563 scope.go:117] "RemoveContainer" containerID="aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae" Jul 29 13:29:36.854424 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:36.854407 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae\": container with ID starting with aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae not found: ID does not exist" containerID="aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae" Jul 29 13:29:36.854523 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.854436 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae"} err="failed to get container status \"aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae\": rpc error: code = NotFound desc = could not find container \"aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae\": container with ID starting with aaec7628c339d9b5abb39720273ce4b562f6aab3de8bfead8196e512f5d5deae not found: ID does not exist" Jul 29 13:29:36.854523 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.854483 2563 scope.go:117] "RemoveContainer" containerID="c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888" Jul 29 13:29:36.854708 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:36.854689 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888\": container with ID starting with c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888 not found: ID does not exist" containerID="c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888" Jul 29 13:29:36.854760 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.854715 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888"} err="failed to get container status \"c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888\": rpc error: code = NotFound desc = could not find container \"c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888\": container with ID starting with c3dd2bf101e9d442b22e9164dacee25edd2bdfdf4fb1b410cea6c2ad06e07888 not found: ID does not exist" Jul 29 13:29:36.859573 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.859552 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82"] Jul 29 13:29:36.863714 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:36.863695 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f0ad55-openshift-default-5ccd86ddc6-5fd82"] Jul 29 13:29:37.659199 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:37.659165 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="936be956-5115-4a68-be8e-95b6c378a023" path="/var/lib/kubelet/pods/936be956-5115-4a68-be8e-95b6c378a023/volumes" Jul 29 13:29:42.331317 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.331283 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm"] Jul 29 13:29:42.331781 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.331546 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" podUID="d1b9a761-7df1-44db-818a-38bae518d3bb" containerName="payload-pre-processing" containerID="cri-o://e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5" gracePeriod=30 Jul 29 13:29:42.582736 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.582677 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:29:42.597224 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.597195 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:42.709949 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.709912 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:42.776105 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.776081 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1b9a761-7df1-44db-818a-38bae518d3bb-plugins-config-volume\") pod \"d1b9a761-7df1-44db-818a-38bae518d3bb\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " Jul 29 13:29:42.776211 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.776110 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9l2x\" (UniqueName: \"kubernetes.io/projected/d1b9a761-7df1-44db-818a-38bae518d3bb-kube-api-access-z9l2x\") pod \"d1b9a761-7df1-44db-818a-38bae518d3bb\" (UID: \"d1b9a761-7df1-44db-818a-38bae518d3bb\") " Jul 29 13:29:42.776561 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.776503 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b9a761-7df1-44db-818a-38bae518d3bb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d1b9a761-7df1-44db-818a-38bae518d3bb" (UID: "d1b9a761-7df1-44db-818a-38bae518d3bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:29:42.778193 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.778171 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b9a761-7df1-44db-818a-38bae518d3bb-kube-api-access-z9l2x" (OuterVolumeSpecName: "kube-api-access-z9l2x") pod "d1b9a761-7df1-44db-818a-38bae518d3bb" (UID: "d1b9a761-7df1-44db-818a-38bae518d3bb"). InnerVolumeSpecName "kube-api-access-z9l2x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:42.874478 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.874382 2563 generic.go:356] "Generic (PLEG): container finished" podID="d1b9a761-7df1-44db-818a-38bae518d3bb" containerID="e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5" exitCode=0 Jul 29 13:29:42.874478 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.874415 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" Jul 29 13:29:42.874631 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.874448 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" event={"ID":"d1b9a761-7df1-44db-818a-38bae518d3bb","Type":"ContainerDied","Data":"e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5"} Jul 29 13:29:42.874631 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.874527 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm" event={"ID":"d1b9a761-7df1-44db-818a-38bae518d3bb","Type":"ContainerDied","Data":"3d518137079d1fbfa51aefc6c2eb01230ed8cde760c1ddd93a3f47167180f676"} Jul 29 13:29:42.874631 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.874557 2563 scope.go:117] "RemoveContainer" containerID="e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5" Jul 29 13:29:42.876845 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.876814 2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1b9a761-7df1-44db-818a-38bae518d3bb-plugins-config-volume\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:42.876845 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.876838 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9l2x\" (UniqueName: \"kubernetes.io/projected/d1b9a761-7df1-44db-818a-38bae518d3bb-kube-api-access-z9l2x\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:42.883504 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.883482 2563 scope.go:117] "RemoveContainer" containerID="e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5" Jul 29 13:29:42.883807 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:42.883789 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5\": container with ID starting with e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5 not found: ID does not exist" containerID="e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5" Jul 29 13:29:42.883879 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.883815 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5"} err="failed to get container status \"e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5\": rpc error: code = NotFound desc = could not find container \"e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5\": container with ID starting with e5e5d9c31482e96b7a697d98ae4c1bfd51b83c16df6cc81d02ffaabe0815d9f5 not found: ID does not exist" Jul 29 13:29:42.898806 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.898785 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm"] Jul 29 13:29:42.903772 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:42.903751 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-11a904-6f7fcbdd68-v4cmm"] Jul 29 13:29:43.004671 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:43.004650 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:43.659877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:43.659846 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1b9a761-7df1-44db-818a-38bae518d3bb" path="/var/lib/kubelet/pods/d1b9a761-7df1-44db-818a-38bae518d3bb/volumes" Jul 29 13:29:49.911640 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:49.911562 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:50.009244 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:50.009211 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:50.201333 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:50.201247 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx"] Jul 29 13:29:50.201838 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:50.201807 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" containerID="cri-o://d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37" gracePeriod=30 Jul 29 13:29:50.219982 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:50.219953 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:50.501080 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:50.501045 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:29:53.609539 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:53.609502 2563 patch_prober.go:28] interesting pod/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 13:29:53.609903 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:53.609556 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 13:29:55.451649 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.451627 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:29:55.475679 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475612 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cd68cc-f687-4be9-8302-7834edf3f935-istiod-ca-cert\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.475795 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475710 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg4q5\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-kube-api-access-bg4q5\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.475795 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475774 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cd68cc-f687-4be9-8302-7834edf3f935-istio-podinfo\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.475900 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475806 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-certs\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.475900 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475841 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-credential-socket\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.475900 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475878 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-data\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.476025 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475914 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-socket\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.476025 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475934 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-envoy\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.476025 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475979 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-istio-token\") pod \"40cd68cc-f687-4be9-8302-7834edf3f935\" (UID: \"40cd68cc-f687-4be9-8302-7834edf3f935\") " Jul 29 13:29:55.476025 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.475988 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40cd68cc-f687-4be9-8302-7834edf3f935-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 13:29:55.476225 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.476208 2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cd68cc-f687-4be9-8302-7834edf3f935-istiod-ca-cert\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.476282 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.476266 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:55.476804 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.476556 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:55.477684 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.477412 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-data" (OuterVolumeSpecName: "istio-data") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:55.477877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.477834 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:55.505380 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.505345 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 13:29:55.505522 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.505384 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40cd68cc-f687-4be9-8302-7834edf3f935-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 29 13:29:55.505522 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.505410 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-istio-token" (OuterVolumeSpecName: "istio-token") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:55.505522 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.505453 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-kube-api-access-bg4q5" (OuterVolumeSpecName: "kube-api-access-bg4q5") pod "40cd68cc-f687-4be9-8302-7834edf3f935" (UID: "40cd68cc-f687-4be9-8302-7834edf3f935"). InnerVolumeSpecName "kube-api-access-bg4q5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 13:29:55.577578 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577557 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg4q5\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-kube-api-access-bg4q5\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577578 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577581 2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cd68cc-f687-4be9-8302-7834edf3f935-istio-podinfo\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577727 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577594 2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-certs\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577727 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577607 2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-credential-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577727 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577618 2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-data\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577727 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577629 2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-workload-socket\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577727 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577640 2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cd68cc-f687-4be9-8302-7834edf3f935-istio-envoy\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.577727 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.577655 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cd68cc-f687-4be9-8302-7834edf3f935-istio-token\") on node \"ip-10-0-137-159.ec2.internal\" DevicePath \"\"" Jul 29 13:29:55.937477 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.937363 2563 generic.go:356] "Generic (PLEG): container finished" podID="40cd68cc-f687-4be9-8302-7834edf3f935" containerID="d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37" exitCode=0 Jul 29 13:29:55.937477 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.937423 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" Jul 29 13:29:55.937477 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.937436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" event={"ID":"40cd68cc-f687-4be9-8302-7834edf3f935","Type":"ContainerDied","Data":"d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37"} Jul 29 13:29:55.937743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.937526 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx" event={"ID":"40cd68cc-f687-4be9-8302-7834edf3f935","Type":"ContainerDied","Data":"d2c2cc7d123cc19e8f557effb03e386d2e731b5eca4afbde3e9150c553e9426a"} Jul 29 13:29:55.937743 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.937553 2563 scope.go:117] "RemoveContainer" containerID="d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37" Jul 29 13:29:55.946846 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.946817 2563 scope.go:117] "RemoveContainer" containerID="2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00" Jul 29 13:29:55.955879 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.955859 2563 scope.go:117] "RemoveContainer" containerID="d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37" Jul 29 13:29:55.956159 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:55.956142 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37\": container with ID starting with d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37 not found: ID does not exist" containerID="d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37" Jul 29 13:29:55.956226 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.956166 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37"} err="failed to get container status \"d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37\": rpc error: code = NotFound desc = could not find container \"d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37\": container with ID starting with d596cc5fb0d3ff88bf10c91a3cfd1bb46b2d8a46d7c77d57573aa668dd38ed37 not found: ID does not exist" Jul 29 13:29:55.956226 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.956184 2563 scope.go:117] "RemoveContainer" containerID="2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00" Jul 29 13:29:55.956357 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.956334 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx"] Jul 29 13:29:55.956401 ip-10-0-137-159 kubenswrapper[2563]: E0729 13:29:55.956375 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00\": container with ID starting with 2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00 not found: ID does not exist" containerID="2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00" Jul 29 13:29:55.956401 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.956393 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00"} err="failed to get container status \"2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00\": rpc error: code = NotFound desc = could not find container \"2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00\": container with ID starting with 2dff73df78a30b2f30382bffb3e1168614ff069e45fb0d895ecc9db09adb0e00 not found: ID does not exist" Jul 29 13:29:55.959146 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:55.959122 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-11a904-openshift-default-747cbbcb56-tphrx"] Jul 29 13:29:56.747891 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:56.747869 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65d577d9d5-86zjt_c8657e2a-3c9c-4b07-a031-1d0a6ae4424c/manager/0.log" Jul 29 13:29:57.244802 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:57.244775 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-j4c42_b4fa787f-e137-47ab-aac6-5236dc7e9bf7/copy-manifests/0.log" Jul 29 13:29:57.265490 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:57.265448 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-j4c42_b4fa787f-e137-47ab-aac6-5236dc7e9bf7/manager/0.log" Jul 29 13:29:57.658328 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:57.658244 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" path="/var/lib/kubelet/pods/40cd68cc-f687-4be9-8302-7834edf3f935/volumes" Jul 29 13:29:59.817655 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:59.817623 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-c6srg_df8bb46a-1aeb-493d-bee8-06f57bebcb0e/manager/0.log" Jul 29 13:29:59.951738 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:29:59.951709 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-c8ss9_5164145e-115e-401f-b187-a31e3a6830a8/manager/0.log" Jul 29 13:30:00.217319 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:00.217297 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6ggkz_2ffe3780-dcad-4f42-8c7e-f102785b5f15/registry-server/0.log" Jul 29 13:30:00.379222 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:00.379196 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qbg8g_597113a7-135f-4935-a8f2-f6b1160c1f51/manager/0.log" Jul 29 13:30:01.002949 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:01.002917 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq_90d1cb53-be29-43d4-b209-15b60beaef6a/istio-proxy/0.log" Jul 29 13:30:01.504454 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:01.504425 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/1.log" Jul 29 13:30:01.632018 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:01.631992 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-2prvh_6e873a36-bb9e-4e02-aaf6-8c19bc718d1d/payload-pre-processing/0.log" Jul 29 13:30:02.330630 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:02.330599 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mp7hj_ac66994e-ce1d-4ed1-97e0-4393e0ab1b15/main/0.log" Jul 29 13:30:02.666931 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:02.666854 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4jkbk_4cad38d4-d1db-4cdd-8d0c-a8a4005adc64/main/0.log" Jul 29 13:30:08.422405 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:08.422370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:30:08.518850 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:08.518814 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" limit=1024 Jul 29 13:30:11.356414 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:11.356385 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lpk5s_a1972259-e332-4fbb-b3db-247f6bdb3f95/global-pull-secret-syncer/0.log" Jul 29 13:30:11.461472 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:11.461428 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cxphl_6ae06bbe-8fd9-437c-b8b0-44e5110c9db5/konnectivity-agent/0.log" Jul 29 13:30:11.587012 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:11.586990 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-159.ec2.internal_a7b4005b800080bf2e43d7f05963a9c0/haproxy/0.log" Jul 29 13:30:16.431454 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:16.431368 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-c6srg_df8bb46a-1aeb-493d-bee8-06f57bebcb0e/manager/0.log" Jul 29 13:30:16.462690 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:16.462667 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-c8ss9_5164145e-115e-401f-b187-a31e3a6830a8/manager/0.log" Jul 29 13:30:16.548432 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:16.548414 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-6ggkz_2ffe3780-dcad-4f42-8c7e-f102785b5f15/registry-server/0.log" Jul 29 13:30:16.670710 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:16.670687 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qbg8g_597113a7-135f-4935-a8f2-f6b1160c1f51/manager/0.log" Jul 29 13:30:18.678712 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:18.678682 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8b4k9_7b0b1a82-9965-4818-9ea7-ceb58b58a0a6/node-exporter/0.log" Jul 29 13:30:18.698667 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:18.698636 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8b4k9_7b0b1a82-9965-4818-9ea7-ceb58b58a0a6/kube-rbac-proxy/0.log" Jul 29 13:30:18.721047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:18.721026 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8b4k9_7b0b1a82-9965-4818-9ea7-ceb58b58a0a6/init-textfile/0.log" Jul 29 13:30:19.006527 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.006500 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/prometheus/0.log" Jul 29 13:30:19.026898 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.026877 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/config-reloader/0.log" Jul 29 13:30:19.049759 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.049738 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/thanos-sidecar/0.log" Jul 29 13:30:19.073764 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.073741 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/kube-rbac-proxy-web/0.log" Jul 29 13:30:19.095997 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.095973 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/kube-rbac-proxy/0.log" Jul 29 13:30:19.120520 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.120498 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/kube-rbac-proxy-thanos/0.log" Jul 29 13:30:19.142102 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:19.142081 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e4bfa46-e9df-4ff6-8e13-21f7854eaffe/init-config-reloader/0.log" Jul 29 13:30:21.371243 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371209 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn"] Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371528 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371539 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371550 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371557 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371567 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b48ec5f-13e8-4734-9546-614d7db6ea27" containerName="payload-processing" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371573 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b48ec5f-13e8-4734-9546-614d7db6ea27" containerName="payload-processing" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371580 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b9a761-7df1-44db-818a-38bae518d3bb" containerName="payload-pre-processing" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371586 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b9a761-7df1-44db-818a-38bae518d3bb" containerName="payload-pre-processing" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371593 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371599 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371606 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371612 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" Jul 29 13:30:21.371613 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371620 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" containerName="payload-pre-processing" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371625 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" containerName="payload-pre-processing" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371631 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371636 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371646 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371651 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371657 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371662 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371706 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371712 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371718 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="40cd68cc-f687-4be9-8302-7834edf3f935" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371725 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b9a761-7df1-44db-818a-38bae518d3bb" containerName="payload-pre-processing" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371729 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e87cebb-139f-4782-b5eb-bbdfe49ebc1b" containerName="payload-pre-processing" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371734 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371739 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371744 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07112c82-a7cd-4eac-bf4b-5b99e76ab248" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371748 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b48ec5f-13e8-4734-9546-614d7db6ea27" containerName="payload-processing" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371754 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="936be956-5115-4a68-be8e-95b6c378a023" containerName="istio-proxy" Jul 29 13:30:21.371995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.371759 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad6115e-9e42-49df-a8ba-8db3c92d566a" containerName="istio-proxy" Jul 29 13:30:21.374931 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.374914 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.377321 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.377289 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gv8zd\"/\"kube-root-ca.crt\"" Jul 29 13:30:21.377321 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.377304 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gv8zd\"/\"default-dockercfg-k884m\"" Jul 29 13:30:21.377993 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.377978 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gv8zd\"/\"openshift-service-ca.crt\"" Jul 29 13:30:21.383547 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.383518 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn"] Jul 29 13:30:21.467394 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.467367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-proc\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.467561 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.467399 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-podres\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.467561 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.467514 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-lib-modules\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.467678 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.467562 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-sys\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.467678 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.467606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh7tf\" (UniqueName: \"kubernetes.io/projected/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-kube-api-access-sh7tf\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.568782 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568754 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-lib-modules\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.568893 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-sys\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.568893 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568837 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh7tf\" (UniqueName: \"kubernetes.io/projected/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-kube-api-access-sh7tf\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.568893 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-proc\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.569047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568891 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-podres\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.569047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568939 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-lib-modules\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.569047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568958 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-proc\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.569047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.568969 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-sys\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.569047 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.569021 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-podres\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.576432 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.576414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh7tf\" (UniqueName: \"kubernetes.io/projected/e2edaf70-ac8f-4c80-acfb-ca3b42b36310-kube-api-access-sh7tf\") pod \"perf-node-gather-daemonset-fpsbn\" (UID: \"e2edaf70-ac8f-4c80-acfb-ca3b42b36310\") " pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.684646 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.684593 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:21.806369 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:21.806340 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn"] Jul 29 13:30:21.808821 ip-10-0-137-159 kubenswrapper[2563]: W0729 13:30:21.808792 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode2edaf70_ac8f_4c80_acfb_ca3b42b36310.slice/crio-d6168d2283e8a51ec5140eab4e6dd86d51037ee81030153295efd7fe5538732c WatchSource:0}: Error finding container d6168d2283e8a51ec5140eab4e6dd86d51037ee81030153295efd7fe5538732c: Status 404 returned error can't find the container with id d6168d2283e8a51ec5140eab4e6dd86d51037ee81030153295efd7fe5538732c Jul 29 13:30:22.052147 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:22.052115 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" event={"ID":"e2edaf70-ac8f-4c80-acfb-ca3b42b36310","Type":"ContainerStarted","Data":"65087b56509a1c94342178eead4a7939a2852c7ba69269dd6f523a63d56d5332"} Jul 29 13:30:22.052308 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:22.052171 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" event={"ID":"e2edaf70-ac8f-4c80-acfb-ca3b42b36310","Type":"ContainerStarted","Data":"d6168d2283e8a51ec5140eab4e6dd86d51037ee81030153295efd7fe5538732c"} Jul 29 13:30:22.052308 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:22.052267 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:22.067981 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:22.067942 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" podStartSLOduration=1.067926603 podStartE2EDuration="1.067926603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 13:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 13:30:22.067496779 +0000 UTC m=+3894.970693448" watchObservedRunningTime="2026-07-29 13:30:22.067926603 +0000 UTC m=+3894.971123281" Jul 29 13:30:23.690863 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:23.690826 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cj9cn_34dabae6-f013-4498-9c47-de54f756de64/dns/0.log" Jul 29 13:30:23.717995 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:23.713194 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cj9cn_34dabae6-f013-4498-9c47-de54f756de64/kube-rbac-proxy/0.log" Jul 29 13:30:23.828589 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:23.828563 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xm56m_700bc436-2f38-40e7-a91d-ac67dad37ed0/dns-node-resolver/0.log" Jul 29 13:30:24.407223 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:24.407191 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9svn6_7e324f8b-44ff-4145-9926-4078115e22cb/node-ca/0.log" Jul 29 13:30:25.369588 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:25.369555 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq_90d1cb53-be29-43d4-b209-15b60beaef6a/istio-proxy/0.log" Jul 29 13:30:25.597960 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:25.597914 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/0.log" Jul 29 13:30:25.650821 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:25.650747 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/1.log" Jul 29 13:30:25.689048 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:25.689024 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-2prvh_6e873a36-bb9e-4e02-aaf6-8c19bc718d1d/payload-pre-processing/0.log" Jul 29 13:30:26.529062 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:26.529030 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4kxgm_d69591be-c441-4afe-8c26-97b512243e07/serve-healthcheck-canary/0.log" Jul 29 13:30:27.179877 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:27.179848 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmgdg_7b259a3c-d681-4dab-a491-38acf064c970/kube-rbac-proxy/0.log" Jul 29 13:30:27.199111 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:27.199090 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmgdg_7b259a3c-d681-4dab-a491-38acf064c970/exporter/0.log" Jul 29 13:30:27.218765 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:27.218738 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmgdg_7b259a3c-d681-4dab-a491-38acf064c970/extractor/0.log" Jul 29 13:30:28.066534 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:28.066512 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gv8zd/perf-node-gather-daemonset-fpsbn" Jul 29 13:30:29.790418 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:29.790384 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-65d577d9d5-86zjt_c8657e2a-3c9c-4b07-a031-1d0a6ae4424c/manager/0.log" Jul 29 13:30:30.085455 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:30.085377 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-j4c42_b4fa787f-e137-47ab-aac6-5236dc7e9bf7/manager/0.log" Jul 29 13:30:30.121768 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:30.121740 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-556f8bf875-j4c42_b4fa787f-e137-47ab-aac6-5236dc7e9bf7/copy-manifests/0.log" Jul 29 13:30:31.573757 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:31.573730 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69d46b4769-lbwdv_58659516-7820-4a11-a02e-47e47a1e1eff/manager/0.log" Jul 29 13:30:31.624591 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:31.624568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq_90d1cb53-be29-43d4-b209-15b60beaef6a/istio-proxy/0.log" Jul 29 13:30:31.801409 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:31.801379 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/0.log" Jul 29 13:30:31.852549 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:31.852530 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/1.log" Jul 29 13:30:31.885684 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:31.885651 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-2prvh_6e873a36-bb9e-4e02-aaf6-8c19bc718d1d/payload-pre-processing/0.log" Jul 29 13:30:33.301515 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:33.301489 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zpnmq_90d1cb53-be29-43d4-b209-15b60beaef6a/istio-proxy/0.log" Jul 29 13:30:33.493141 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:33.493111 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/0.log" Jul 29 13:30:33.543005 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:33.542975 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-fvg79_aca93527-1778-4111-b825-9069d2b0fb94/istio-proxy/1.log" Jul 29 13:30:33.574396 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:33.574335 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-2prvh_6e873a36-bb9e-4e02-aaf6-8c19bc718d1d/payload-pre-processing/0.log" Jul 29 13:30:36.588845 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:36.588814 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-kf9vv_44d62cfa-a814-4009-8e9e-002507edae0a/migrator/0.log" Jul 29 13:30:36.615595 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:36.615568 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-kf9vv_44d62cfa-a814-4009-8e9e-002507edae0a/graceful-termination/0.log" Jul 29 13:30:37.962882 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:37.962853 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/kube-multus-additional-cni-plugins/0.log" Jul 29 13:30:37.987575 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:37.987552 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/egress-router-binary-copy/0.log" Jul 29 13:30:38.012796 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:38.012774 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/cni-plugins/0.log" Jul 29 13:30:38.037622 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:38.037602 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/bond-cni-plugin/0.log" Jul 29 13:30:38.058980 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:38.058960 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/routeoverride-cni/0.log" Jul 29 13:30:38.082380 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:38.082363 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/whereabouts-cni-bincopy/0.log" Jul 29 13:30:38.105853 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:38.105834 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6vqb2_c9942c8c-5762-439f-8ce1-8bd261960a9e/whereabouts-cni/0.log" Jul 29 13:30:38.528450 ip-10-0-137-159 kubenswrapper[2563]: I0729 13:30:38.528425 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-x4qbv_6d292488-7717-4073-9f75-5c47539006d6/kube-multus/0.log"