Jul 24 13:09:31.354195 ip-10-0-137-131 systemd[1]: Starting Kubernetes Kubelet... Jul 24 13:09:31.660358 ip-10-0-137-131 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:09:31.660358 ip-10-0-137-131 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 13:09:31.660358 ip-10-0-137-131 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:09:31.660358 ip-10-0-137-131 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 13:09:31.660358 ip-10-0-137-131 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 13:09:31.661022 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.660348 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 13:09:31.664943 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664915 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:09:31.664943 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664943 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664947 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664950 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664954 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664957 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664960 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664962 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664965 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664968 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664971 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664973 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664976 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664978 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664981 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664984 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664987 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664989 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.664997 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665000 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665002 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:09:31.665007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665005 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665008 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665012 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665017 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665021 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665024 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665027 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665031 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665034 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665037 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665040 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665042 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665045 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665047 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665050 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665052 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665055 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665058 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665060 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:09:31.665470 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665063 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665065 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665067 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665072 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665075 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665078 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665080 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665083 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665085 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665088 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665090 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665093 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665096 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665098 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665101 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665104 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665107 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665110 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665112 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665115 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:09:31.665954 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665117 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665120 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665123 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665125 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665127 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665130 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665133 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665135 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665138 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665140 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665143 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665146 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665148 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665151 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665153 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665156 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665159 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665161 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665164 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665166 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:09:31.666431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665169 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665171 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665174 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665177 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665181 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665184 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665188 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665192 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665195 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665198 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665200 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665203 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665205 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665208 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665210 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665213 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665215 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665218 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665221 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:09:31.666900 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665940 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665948 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665951 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665954 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665956 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665959 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665962 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665965 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665968 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665970 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665973 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665976 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665979 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665981 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665984 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665987 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665990 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665992 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665995 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:09:31.667349 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.665997 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666001 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666003 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666006 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666009 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666011 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666014 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666016 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666019 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666022 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666024 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666027 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666030 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666032 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666035 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666037 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666040 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666042 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666045 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666047 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:09:31.667807 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666050 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666053 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666055 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666058 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666063 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666067 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666070 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666073 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666076 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666079 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666082 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666084 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666087 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666089 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666092 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666095 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666097 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666100 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666103 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:09:31.668338 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666105 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666108 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666111 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666113 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666116 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666119 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666121 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666124 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666127 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666129 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666131 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666134 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666137 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666139 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666142 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666144 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666147 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666150 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666153 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666156 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:09:31.668808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666158 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666161 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666163 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666166 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666171 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666174 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666177 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666182 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666185 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666189 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666191 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666195 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666198 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666200 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666204 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666208 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666212 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666215 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666217 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:09:31.669297 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666220 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666223 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666599 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666605 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666609 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666612 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666615 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666617 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666620 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666623 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666626 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666629 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666632 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666635 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666637 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666640 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666642 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666645 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666648 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:09:31.669783 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666650 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666653 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666657 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666659 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666662 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666665 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666667 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666670 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666673 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666675 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666679 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666681 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666684 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666686 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666689 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666692 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666694 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666696 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666699 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:09:31.670260 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666701 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666704 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666706 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666709 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666712 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666714 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666717 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666719 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666722 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666724 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666727 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666729 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666731 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666734 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666737 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666740 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666742 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666745 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666747 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666750 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:09:31.670715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666753 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666756 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666758 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666761 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666763 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666766 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666768 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666771 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666773 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666776 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666778 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666782 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666785 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666788 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666790 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666793 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666795 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666798 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666800 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666803 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:09:31.671240 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666805 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666808 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666811 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666813 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666816 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666819 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666821 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666824 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666826 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666829 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666832 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666834 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666837 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666839 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666841 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666844 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666846 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666849 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666852 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:09:31.671720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666856 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666859 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666862 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.666865 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666958 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666968 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666979 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666984 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666987 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666991 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666995 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.666999 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667002 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667006 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667010 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667013 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667016 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667021 2562 flags.go:64] FLAG: --cgroup-root="" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667024 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667027 2562 flags.go:64] FLAG: --client-ca-file="" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667030 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667033 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667037 2562 flags.go:64] FLAG: --cluster-domain="" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667039 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 13:09:31.672185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667042 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667045 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667049 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667052 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667055 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667058 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667061 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667064 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667067 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667070 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667073 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667076 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667080 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667083 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667086 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667089 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667091 2562 flags.go:64] FLAG: --enable-server="true" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667094 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667099 2562 flags.go:64] FLAG: --event-burst="100" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667102 2562 flags.go:64] FLAG: --event-qps="50" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667105 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667108 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667111 2562 flags.go:64] FLAG: --eviction-hard="" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667115 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 13:09:31.672769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667118 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667122 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667126 2562 flags.go:64] FLAG: --eviction-soft="" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667128 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667131 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667134 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667137 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667140 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667143 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667146 2562 flags.go:64] FLAG: --feature-gates="" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667149 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667152 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667155 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667159 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667161 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667164 2562 flags.go:64] FLAG: --help="false" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667167 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-137-131.ec2.internal" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667170 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667173 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667176 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667179 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667182 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667185 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 13:09:31.673353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667188 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667191 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667193 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667196 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667200 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667202 2562 flags.go:64] FLAG: --kube-reserved="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667205 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667209 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667213 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667216 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667221 2562 flags.go:64] FLAG: --lock-file="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667224 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667226 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667229 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667235 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667238 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667240 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667243 2562 flags.go:64] FLAG: --logging-format="text" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667246 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667249 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667252 2562 flags.go:64] FLAG: --manifest-url="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667255 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667260 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667263 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667267 2562 flags.go:64] FLAG: --max-pods="110" Jul 24 13:09:31.673954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667270 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667273 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667276 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667278 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667281 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667284 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667287 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667293 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667297 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667300 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667302 2562 flags.go:64] FLAG: --pod-cidr="" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667305 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667314 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667317 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667320 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667325 2562 flags.go:64] FLAG: --port="10250" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667329 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667334 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0de5881669a3d6283" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667337 2562 flags.go:64] FLAG: --qos-reserved="" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667340 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667343 2562 flags.go:64] FLAG: --register-node="true" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667346 2562 flags.go:64] FLAG: --register-with-taints="" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667349 2562 flags.go:64] FLAG: --registry-burst="10" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667352 2562 flags.go:64] FLAG: --registry-qps="5" Jul 24 13:09:31.674830 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667355 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667358 2562 flags.go:64] FLAG: --reserved-memory="" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667361 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667365 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667368 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667370 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667373 2562 flags.go:64] FLAG: --runonce="false" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667376 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667379 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667382 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667385 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667388 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667391 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667394 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667397 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667400 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667403 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667406 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667409 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667411 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667414 2562 flags.go:64] FLAG: --system-cgroups="" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667417 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667422 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667426 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667429 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 13:09:31.675540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667435 2562 flags.go:64] FLAG: --tls-min-version="" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667438 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667441 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667443 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667447 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667450 2562 flags.go:64] FLAG: --v="2" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667453 2562 flags.go:64] FLAG: --version="false" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667457 2562 flags.go:64] FLAG: --vmodule="" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667460 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.667464 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667578 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667582 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667585 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667589 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667592 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667595 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667597 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667600 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667604 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667608 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667611 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667614 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:09:31.676164 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667617 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667620 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667622 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667625 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667628 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667631 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667634 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667636 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667640 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667643 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667647 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667650 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667653 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667655 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667658 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667660 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667663 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667666 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667668 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667671 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:09:31.676712 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667673 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667677 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667680 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667682 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667685 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667688 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667690 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667692 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667695 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667698 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667700 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667704 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667708 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667710 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667714 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667716 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667719 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667721 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667723 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:09:31.677255 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667726 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667730 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667733 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667736 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667739 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667743 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667746 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667748 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667751 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667753 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667756 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667759 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667761 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667764 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667766 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667769 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667771 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667774 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667776 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:09:31.677715 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667779 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667781 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667784 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667787 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667789 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667792 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667795 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667797 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667800 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667802 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667805 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667808 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667811 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667814 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667818 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667820 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667824 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667827 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.667829 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668205 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:09:31.678197 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668208 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668211 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668213 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668216 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668219 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668222 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668224 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668227 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.668229 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.668929 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.676794 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.676810 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.676830 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.676835 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 13:09:31.678666 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676891 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676897 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676901 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676904 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676908 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676911 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676914 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676932 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676936 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676939 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676941 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676944 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676948 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676952 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676955 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676958 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676961 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676963 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676966 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676968 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:09:31.679033 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676971 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676973 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676977 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676979 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676982 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676985 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676987 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676990 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676992 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676996 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.676998 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677001 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677004 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677006 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677009 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677012 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677015 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677017 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677020 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677023 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:09:31.679509 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677025 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677028 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677030 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677033 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677035 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677037 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677040 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677042 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677045 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677047 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677050 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677053 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677055 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677058 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677061 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677063 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677066 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677069 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677071 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677073 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:09:31.680007 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677076 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677079 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677082 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677085 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677087 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677090 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677092 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677095 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677097 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677100 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677103 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677105 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677107 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677112 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677115 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677118 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677121 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677123 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677126 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:09:31.680486 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677130 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677132 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677135 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677137 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677140 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677142 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677145 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677147 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677150 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677152 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677155 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677157 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677159 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677162 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677165 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677168 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677171 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677173 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677176 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:09:31.680950 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677178 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.677183 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677294 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677299 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677302 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677305 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677308 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677311 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677315 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677319 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677322 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677325 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677328 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677330 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 13:09:31.681418 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677333 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677337 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677341 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677344 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677347 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677350 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677353 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677356 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677358 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677361 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677363 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677366 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677368 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677372 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677375 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677378 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677380 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677383 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677385 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 13:09:31.681764 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677388 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677391 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677393 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677396 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677398 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677400 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677403 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677406 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677408 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677410 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677413 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677416 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677418 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677421 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677425 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677428 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677431 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677434 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677437 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 13:09:31.682306 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677440 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677442 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677445 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677447 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677450 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677452 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677455 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677457 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677460 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677462 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677465 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677467 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677470 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677472 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677475 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677478 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677480 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677483 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677485 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677488 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 13:09:31.682759 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677490 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677492 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677495 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677498 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677500 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677502 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677505 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677507 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677511 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677513 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677516 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677518 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677521 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677523 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677526 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677528 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677531 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677533 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677536 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677538 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 13:09:31.683241 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677541 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677543 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677546 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677548 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677551 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677553 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677556 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677558 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:31.677561 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.677565 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.677659 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.680775 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.681879 2562 server.go:1019] "Starting client certificate rotation" Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.681967 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:09:31.683704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.682008 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 13:09:31.684574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.684559 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:09:31.688366 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.688349 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 13:09:31.696433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.696417 2562 log.go:25] "Validated CRI v1 runtime API" Jul 24 13:09:31.698270 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.698255 2562 log.go:25] "Validated CRI v1 image API" Jul 24 13:09:31.699609 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.699591 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 13:09:31.701606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.701589 2562 fs.go:135] Filesystem UUIDs: map[5152c832-5dab-40fe-8679-1693fc341131:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cfdb9859-3e5c-4dda-8a23-5accb12179ed:/dev/nvme0n1p4] Jul 24 13:09:31.701678 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.701606 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 13:09:31.707520 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.707405 2562 manager.go:216] Machine: {Timestamp:2026-07-24 13:09:31.706002299 +0000 UTC m=+0.260873460 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3110140 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26d0ef6755e57aa3039ca934f810d7 SystemUUID:ec26d0ef-6755-e57a-a303-9ca934f810d7 BootID:4a9ece14-9f50-40cb-9be4-22ef605b3326 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c0:9f:ec:c6:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c0:9f:ec:c6:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:0d:16:45:9a:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 13:09:31.707520 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.707509 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 13:09:31.707661 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.707589 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 13:09:31.709883 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.709857 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 13:09:31.710027 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.709883 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-131.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 13:09:31.710464 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.710454 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 13:09:31.710509 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.710467 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 13:09:31.710509 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.710474 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:09:31.710952 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.710942 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 13:09:31.711687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.711678 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 13:09:31.713144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.713129 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:09:31.714271 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714260 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:09:31.714382 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714372 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 13:09:31.714420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714401 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 24 13:09:31.714420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714411 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 13:09:31.714474 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714426 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 13:09:31.714474 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714435 2562 kubelet.go:398] "Adding apiserver pod source" Jul 24 13:09:31.714474 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714444 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 13:09:31.714774 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714762 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:09:31.714839 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.714780 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 13:09:31.716441 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.716424 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 13:09:31.718064 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.718048 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 13:09:31.718139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.718086 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 13:09:31.719362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719348 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719367 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719376 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719383 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719395 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719404 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719415 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719423 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 13:09:31.719432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719433 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 13:09:31.719692 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719442 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 13:09:31.719692 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719457 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 13:09:31.719692 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.719465 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 13:09:31.720018 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.720007 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 13:09:31.720066 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.720021 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 13:09:31.723262 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.723248 2562 server.go:1268] "Started kubelet" Jul 24 13:09:31.723417 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.723392 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 13:09:31.723491 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.723375 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 13:09:31.723558 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.723537 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 13:09:31.723970 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.723950 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-131.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 13:09:31.724114 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.724095 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 13:09:31.724186 ip-10-0-137-131 systemd[1]: Started Kubernetes Kubelet. Jul 24 13:09:31.724554 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.724542 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 24 13:09:31.725153 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.724996 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 13:09:31.725255 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.725230 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 13:09:31.727939 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.727908 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 13:09:31.728020 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.727309 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-131.ec2.internal.18c53bcb4faa3718 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-131.ec2.internal,UID:ip-10-0-137-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-131.ec2.internal,},FirstTimestamp:2026-07-24 13:09:31.723224856 +0000 UTC m=+0.278096013,LastTimestamp:2026-07-24 13:09:31.723224856 +0000 UTC m=+0.278096013,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-131.ec2.internal,}" Jul 24 13:09:31.730768 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.730752 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 13:09:31.730768 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.730765 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 13:09:31.731195 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.730899 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:31.731482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731393 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 13:09:31.731482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731411 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 13:09:31.731482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731468 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 13:09:31.731650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731493 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 24 13:09:31.731650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731501 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 24 13:09:31.731650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731569 2562 factory.go:55] Registering systemd factory Jul 24 13:09:31.731650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731580 2562 factory.go:223] Registration of the systemd container factory successfully Jul 24 13:09:31.731884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731782 2562 factory.go:153] Registering CRI-O factory Jul 24 13:09:31.731884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731794 2562 factory.go:223] Registration of the crio container factory successfully Jul 24 13:09:31.731884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731844 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 13:09:31.731884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731864 2562 factory.go:103] Registering Raw factory Jul 24 13:09:31.731884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.731875 2562 manager.go:1196] Started watching for new ooms in manager Jul 24 13:09:31.732660 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.732632 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 13:09:31.732757 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.732710 2562 manager.go:349] Starting recovery of all containers Jul 24 13:09:31.732757 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:31.732721 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 13:09:31.740960 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.740930 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tzx5b" Jul 24 13:09:31.745100 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.745087 2562 manager.go:354] Recovery completed Jul 24 13:09:31.749440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.749428 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:31.749767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.749752 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tzx5b" Jul 24 13:09:31.751288 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751269 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:31.751356 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751306 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:31.751356 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751321 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:31.751835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751812 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 13:09:31.751835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751825 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 13:09:31.751971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751839 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 24 13:09:31.751971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.751870 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:31.752610 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.752593 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:31.752660 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.752626 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:31.752660 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.752642 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:31.754483 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.754470 2562 policy_none.go:49] "None policy: Start" Jul 24 13:09:31.754551 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.754488 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 13:09:31.754551 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.754501 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 13:09:31.756754 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:31.756738 2562 policy_none.go:47] "Start" Jul 24 13:09:33.759218 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759043 2562 manager.go:337] "Starting Device Plugin manager" Jul 24 13:09:33.759310 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.759236 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 13:09:33.759310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759251 2562 server.go:87] "Starting device plugin registration server" Jul 24 13:09:33.759490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759478 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 13:09:33.759536 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759490 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 13:09:33.759674 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759611 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 13:09:33.759731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759710 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 13:09:33.759731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.759720 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 13:09:33.760194 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.760175 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 13:09:33.760288 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.760228 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:33.859780 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.859743 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:33.861942 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.860627 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:33.861942 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.860656 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:33.861942 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.860665 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:33.861942 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.860691 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-131.ec2.internal" Jul 24 13:09:33.870926 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.870880 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-131.ec2.internal" Jul 24 13:09:33.870926 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.870900 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-131.ec2.internal\": node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:33.886986 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.886958 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:33.888410 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.888388 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 13:09:33.889663 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.889644 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 13:09:33.889742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.889688 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 13:09:33.889742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.889714 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 13:09:33.889833 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.889747 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 13:09:33.894271 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.894257 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:09:33.987360 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:33.987319 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:33.990602 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.990581 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal"] Jul 24 13:09:33.990664 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.990645 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:33.992033 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.992017 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:33.992118 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.992047 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:33.992118 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.992088 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:33.994767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.994751 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:33.994884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.994867 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" Jul 24 13:09:33.994925 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.994907 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:33.995460 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.995442 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:33.995525 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.995477 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:33.995525 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.995487 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:33.995525 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.995451 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:33.995682 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.995562 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:33.995682 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.995585 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:33.997711 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.997696 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:33.997792 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.997721 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 13:09:33.998339 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.998323 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientMemory" Jul 24 13:09:33.998436 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.998346 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 13:09:33.998436 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:33.998359 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeHasSufficientPID" Jul 24 13:09:34.021247 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.021228 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-131.ec2.internal\" not found" node="ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.025504 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.025489 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-131.ec2.internal\" not found" node="ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.088038 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.088018 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.089142 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.089125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be7a3e104963d2d6fa190dbf68a99d50-config\") pod \"kube-apiserver-proxy-ip-10-0-137-131.ec2.internal\" (UID: \"be7a3e104963d2d6fa190dbf68a99d50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.089196 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.089149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b343ceb436b78e78994571ea8397deb4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal\" (UID: \"b343ceb436b78e78994571ea8397deb4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.089196 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.089168 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b343ceb436b78e78994571ea8397deb4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal\" (UID: \"b343ceb436b78e78994571ea8397deb4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.188656 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.188592 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.189706 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.189691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b343ceb436b78e78994571ea8397deb4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal\" (UID: \"b343ceb436b78e78994571ea8397deb4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.189757 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.189716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b343ceb436b78e78994571ea8397deb4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal\" (UID: \"b343ceb436b78e78994571ea8397deb4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.189757 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.189732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be7a3e104963d2d6fa190dbf68a99d50-config\") pod \"kube-apiserver-proxy-ip-10-0-137-131.ec2.internal\" (UID: \"be7a3e104963d2d6fa190dbf68a99d50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.189821 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.189775 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/be7a3e104963d2d6fa190dbf68a99d50-config\") pod \"kube-apiserver-proxy-ip-10-0-137-131.ec2.internal\" (UID: \"be7a3e104963d2d6fa190dbf68a99d50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.189821 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.189797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b343ceb436b78e78994571ea8397deb4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal\" (UID: \"b343ceb436b78e78994571ea8397deb4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.189821 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.189798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b343ceb436b78e78994571ea8397deb4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal\" (UID: \"b343ceb436b78e78994571ea8397deb4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.289063 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.289027 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.324290 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.324272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.329049 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.329003 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:34.389423 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.389397 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.489886 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.489827 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.590249 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.590220 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.599905 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.599886 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:09:34.636982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.636958 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 13:09:34.637429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.637076 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:09:34.637429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.637078 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 13:09:34.690907 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.690887 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.712150 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.712106 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 13:04:33 +0000 UTC" deadline="2028-02-02 03:54:20.578634585 +0000 UTC" Jul 24 13:09:34.712150 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.712144 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13382h44m45.866493484s" Jul 24 13:09:34.791670 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.791598 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.812279 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.812257 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:09:34.891706 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.891685 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:34.970399 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:34.970369 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb343ceb436b78e78994571ea8397deb4.slice/crio-bdfd78c237f6b3faa8952c60365788b75a0cb0b99eba8bbdbda3aef9fc079e7e WatchSource:0}: Error finding container bdfd78c237f6b3faa8952c60365788b75a0cb0b99eba8bbdbda3aef9fc079e7e: Status 404 returned error can't find the container with id bdfd78c237f6b3faa8952c60365788b75a0cb0b99eba8bbdbda3aef9fc079e7e Jul 24 13:09:34.971082 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:34.971063 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7a3e104963d2d6fa190dbf68a99d50.slice/crio-2718bd10c8941879003984299b33821404be65d631696b3e5bc6e13e08c1a3f5 WatchSource:0}: Error finding container 2718bd10c8941879003984299b33821404be65d631696b3e5bc6e13e08c1a3f5: Status 404 returned error can't find the container with id 2718bd10c8941879003984299b33821404be65d631696b3e5bc6e13e08c1a3f5 Jul 24 13:09:34.974896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:34.974879 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:09:34.992209 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:34.992188 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:35.092745 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.092658 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:35.193453 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.193413 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-131.ec2.internal\" not found" Jul 24 13:09:35.196719 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.196702 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:09:35.287539 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.287511 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" Jul 24 13:09:35.301198 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.301175 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:09:35.302105 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.302091 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" Jul 24 13:09:35.312935 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.312915 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 13:09:35.611087 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.611048 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:09:35.671232 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.671200 2562 apiserver.go:52] "Watching apiserver" Jul 24 13:09:35.677720 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.677697 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 13:09:35.679503 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.679453 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-g5525","openshift-multus/multus-48gd6","openshift-multus/network-metrics-daemon-7tttb","openshift-network-operator/iptables-alerter-7c8qv","kube-system/global-pull-secret-syncer-dv8tx","openshift-image-registry/node-ca-2z5z2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal","openshift-multus/multus-additional-cni-plugins-vcb8n","openshift-network-diagnostics/network-check-target-8ncd6","openshift-ovn-kubernetes/ovnkube-node-dl6cj","kube-system/konnectivity-agent-xkb2z","kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb","openshift-cluster-node-tuning-operator/tuned-tpbww"] Jul 24 13:09:35.682281 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.682259 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.685036 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685016 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.685144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685124 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 13:09:35.685280 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685262 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.685367 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685342 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 13:09:35.685602 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685531 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hwmw9\"" Jul 24 13:09:35.685696 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685622 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.685757 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685537 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 13:09:35.685898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.685871 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 13:09:35.687726 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.686980 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 13:09:35.689175 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.689155 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.689295 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.689279 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.690386 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.689685 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.690386 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.690157 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fk6cv\"" Jul 24 13:09:35.690386 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.690311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.693320 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.692737 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.694475 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.693972 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.694475 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.694054 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:35.694710 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.694530 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 13:09:35.695515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.695220 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 13:09:35.695515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.695326 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 13:09:35.695515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.695365 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.695515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.695423 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d4szw\"" Jul 24 13:09:35.695515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.695505 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nw4tq\"" Jul 24 13:09:35.696799 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.696221 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.698038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-cni-bin\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698067 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-modprobe-d\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698094 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-var-lib-kubelet\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24998053-2978-4cb6-8410-6a83cf7418e9-tmp\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx4gw\" (UniqueName: \"kubernetes.io/projected/24998053-2978-4cb6-8410-6a83cf7418e9-kube-api-access-cx4gw\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-kubelet\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-systemd-units\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-ovn\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698229 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-run-ovn-kubernetes\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698253 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovn-node-metrics-cert\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-sys\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698332 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c393e7a-a2e8-416d-91e8-b18832ac181b-agent-certs\") pod \"konnectivity-agent-xkb2z\" (UID: \"0c393e7a-a2e8-416d-91e8-b18832ac181b\") " pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8k2b\" (UniqueName: \"kubernetes.io/projected/2d35d896-41aa-41e0-ad1e-456449f5a4a8-kube-api-access-x8k2b\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-etc-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698401 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698422 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysctl-d\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698445 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovnkube-config\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-run\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-kubernetes\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysconfig\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-systemd\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-host\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24998053-2978-4cb6-8410-6a83cf7418e9-etc-tuned\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-slash\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-cni-netd\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.698684 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698683 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698700 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-lib-modules\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d35d896-41aa-41e0-ad1e-456449f5a4a8-host-slash\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-systemd\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698804 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-env-overrides\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovnkube-script-lib\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698855 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d35d896-41aa-41e0-ad1e-456449f5a4a8-iptables-alerter-script\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-var-lib-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d68r\" (UniqueName: \"kubernetes.io/projected/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-kube-api-access-4d68r\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysctl-conf\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.698958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c393e7a-a2e8-416d-91e8-b18832ac181b-konnectivity-ca\") pod \"konnectivity-agent-xkb2z\" (UID: \"0c393e7a-a2e8-416d-91e8-b18832ac181b\") " pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-run-netns\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699074 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-node-log\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699082 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-log-socket\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699289 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z956h\"" Jul 24 13:09:35.699371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699343 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 13:09:35.700182 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.699589 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.700948 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.700928 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:35.701051 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.700995 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:35.702521 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.702502 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 13:09:35.703160 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.703140 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2w8tr\"" Jul 24 13:09:35.703373 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.703356 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.703869 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.703852 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 13:09:35.703948 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.703897 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.704001 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.703943 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.704069 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.703856 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 13:09:35.706187 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.706168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:35.706273 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.706207 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q579w\"" Jul 24 13:09:35.706273 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.706228 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:35.706381 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.706305 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.706620 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.706603 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.708795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.708776 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.711127 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.710980 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 13:09:35.711391 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.711374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 13:09:35.712294 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.712275 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.712375 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.712299 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 13:09:35.712375 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.712315 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7svfn\"" Jul 24 13:09:35.714255 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.714237 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 13:09:35.714495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.714481 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qtmgd\"" Jul 24 13:09:35.734450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.734424 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kh2bf" Jul 24 13:09:35.742698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.742671 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kh2bf" Jul 24 13:09:35.789175 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.789154 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 13:09:35.800247 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800268 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysctl-d\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.800359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rs7p\" (UniqueName: \"kubernetes.io/projected/9ce1fb28-3c70-446e-88a6-55447582c3ad-kube-api-access-9rs7p\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.800359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800327 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8cpr\" (UniqueName: \"kubernetes.io/projected/86f4acca-d628-4c62-986a-f66e371b9aa4-kube-api-access-b8cpr\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.800359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-multus-certs\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800373 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-registration-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2e52ad7-bbac-4736-9140-0ce86767eaf2-cni-binary-copy\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-kubelet\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-kubernetes\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800509 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysconfig\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.800574 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-systemd\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24998053-2978-4cb6-8410-6a83cf7418e9-etc-tuned\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62s5z\" (UniqueName: \"kubernetes.io/projected/89f9861e-d807-40b0-ba0f-3e22c35832bd-kube-api-access-62s5z\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-netns\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800655 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800697 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-slash\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800720 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-cni-netd\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysctl-d\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-env-overrides\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovnkube-script-lib\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d35d896-41aa-41e0-ad1e-456449f5a4a8-iptables-alerter-script\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-sys-fs\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.800997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-kubernetes\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-k8s-cni-cncf-io\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-daemon-config\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800961 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d68r\" (UniqueName: \"kubernetes.io/projected/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-kube-api-access-4d68r\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.800995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c393e7a-a2e8-416d-91e8-b18832ac181b-konnectivity-ca\") pod \"konnectivity-agent-xkb2z\" (UID: \"0c393e7a-a2e8-416d-91e8-b18832ac181b\") " pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801018 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-cni-multus\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-cni-bin\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-systemd\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-var-lib-kubelet\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801097 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24998053-2978-4cb6-8410-6a83cf7418e9-tmp\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx4gw\" (UniqueName: \"kubernetes.io/projected/24998053-2978-4cb6-8410-6a83cf7418e9-kube-api-access-cx4gw\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8k2b\" (UniqueName: \"kubernetes.io/projected/2d35d896-41aa-41e0-ad1e-456449f5a4a8-kube-api-access-x8k2b\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/977bf748-3237-46dd-8171-f882894b111d-serviceca\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801194 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djnhn\" (UniqueName: \"kubernetes.io/projected/977bf748-3237-46dd-8171-f882894b111d-kube-api-access-djnhn\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801226 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-os-release\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-kubelet\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.801781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-run-ovn-kubernetes\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovn-node-metrics-cert\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-sys\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801380 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-cni-binary-copy\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-cnibin\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-cni-bin\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-etc-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801516 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/277edfa4-9e9a-4177-b38d-1bdffa4457ed-dbus\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovnkube-config\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-run\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-conf-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801656 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-host\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-os-release\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801715 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801722 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-var-lib-kubelet\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.802578 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-system-cni-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801751 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-sys\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-socket-dir-parent\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-hostroot\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801802 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-slash\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-lib-modules\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-cni-netd\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d35d896-41aa-41e0-ad1e-456449f5a4a8-host-slash\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801892 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-cnibin\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-systemd\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6qw7\" (UniqueName: \"kubernetes.io/projected/79c34749-5c39-4255-8891-20979a6cff44-kube-api-access-d6qw7\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802046 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-etc-kubernetes\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ggvx\" (UniqueName: \"kubernetes.io/projected/a2e52ad7-bbac-4736-9140-0ce86767eaf2-kube-api-access-4ggvx\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802103 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-var-lib-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysctl-conf\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/977bf748-3237-46dd-8171-f882894b111d-host\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802224 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-socket-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.803359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-var-lib-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802349 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-env-overrides\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802420 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysctl-conf\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802451 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-lib-modules\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802458 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-etc-openvswitch\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802489 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d35d896-41aa-41e0-ad1e-456449f5a4a8-host-slash\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-systemd\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-device-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802626 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-run-netns\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-node-log\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-log-socket\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802704 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-modprobe-d\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c393e7a-a2e8-416d-91e8-b18832ac181b-agent-certs\") pod \"konnectivity-agent-xkb2z\" (UID: \"0c393e7a-a2e8-416d-91e8-b18832ac181b\") " pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-system-cni-dir\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86f4acca-d628-4c62-986a-f66e371b9aa4-tmp-dir\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802825 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/277edfa4-9e9a-4177-b38d-1bdffa4457ed-kubelet-config\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802841 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovnkube-script-lib\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-systemd-units\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-ovn\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86f4acca-d628-4c62-986a-f66e371b9aa4-hosts-file\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.802999 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-cni-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803059 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-run\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-systemd-units\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-run-ovn\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803205 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-kubelet\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovnkube-config\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803210 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-host\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803268 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-run-ovn-kubernetes\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803312 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-node-log\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d35d896-41aa-41e0-ad1e-456449f5a4a8-iptables-alerter-script\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803373 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-run-netns\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803411 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-log-socket\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803417 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-modprobe-d\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.801020 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24998053-2978-4cb6-8410-6a83cf7418e9-etc-sysconfig\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.804917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803485 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-host-cni-bin\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.805822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.803986 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0c393e7a-a2e8-416d-91e8-b18832ac181b-konnectivity-ca\") pod \"konnectivity-agent-xkb2z\" (UID: \"0c393e7a-a2e8-416d-91e8-b18832ac181b\") " pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.807086 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.807064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24998053-2978-4cb6-8410-6a83cf7418e9-tmp\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.807896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.807868 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24998053-2978-4cb6-8410-6a83cf7418e9-etc-tuned\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.808308 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.808286 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0c393e7a-a2e8-416d-91e8-b18832ac181b-agent-certs\") pod \"konnectivity-agent-xkb2z\" (UID: \"0c393e7a-a2e8-416d-91e8-b18832ac181b\") " pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:35.810044 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.810021 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-ovn-node-metrics-cert\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.815529 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.815507 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx4gw\" (UniqueName: \"kubernetes.io/projected/24998053-2978-4cb6-8410-6a83cf7418e9-kube-api-access-cx4gw\") pod \"tuned-tpbww\" (UID: \"24998053-2978-4cb6-8410-6a83cf7418e9\") " pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:35.819563 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.819528 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d68r\" (UniqueName: \"kubernetes.io/projected/a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3-kube-api-access-4d68r\") pod \"ovnkube-node-dl6cj\" (UID: \"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3\") " pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:35.819975 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.819954 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8k2b\" (UniqueName: \"kubernetes.io/projected/2d35d896-41aa-41e0-ad1e-456449f5a4a8-kube-api-access-x8k2b\") pod \"iptables-alerter-7c8qv\" (UID: \"2d35d896-41aa-41e0-ad1e-456449f5a4a8\") " pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:35.895890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.895795 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" event={"ID":"b343ceb436b78e78994571ea8397deb4","Type":"ContainerStarted","Data":"bdfd78c237f6b3faa8952c60365788b75a0cb0b99eba8bbdbda3aef9fc079e7e"} Jul 24 13:09:35.897203 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.897157 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" event={"ID":"be7a3e104963d2d6fa190dbf68a99d50","Type":"ContainerStarted","Data":"2718bd10c8941879003984299b33821404be65d631696b3e5bc6e13e08c1a3f5"} Jul 24 13:09:35.903641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903618 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-hostroot\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.903762 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-cnibin\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.903762 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6qw7\" (UniqueName: \"kubernetes.io/projected/79c34749-5c39-4255-8891-20979a6cff44-kube-api-access-d6qw7\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:35.903762 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-etc-kubernetes\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.903762 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ggvx\" (UniqueName: \"kubernetes.io/projected/a2e52ad7-bbac-4736-9140-0ce86767eaf2-kube-api-access-4ggvx\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/977bf748-3237-46dd-8171-f882894b111d-host\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-socket-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903791 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-cnibin\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-device-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903846 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-etc-kubernetes\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903849 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-hostroot\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-system-cni-dir\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903870 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-device-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903910 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/977bf748-3237-46dd-8171-f882894b111d-host\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86f4acca-d628-4c62-986a-f66e371b9aa4-tmp-dir\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903931 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-system-cni-dir\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903944 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/277edfa4-9e9a-4177-b38d-1bdffa4457ed-kubelet-config\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.903963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86f4acca-d628-4c62-986a-f66e371b9aa4-hosts-file\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.903997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-cni-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rs7p\" (UniqueName: \"kubernetes.io/projected/9ce1fb28-3c70-446e-88a6-55447582c3ad-kube-api-access-9rs7p\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904042 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-socket-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8cpr\" (UniqueName: \"kubernetes.io/projected/86f4acca-d628-4c62-986a-f66e371b9aa4-kube-api-access-b8cpr\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-multus-certs\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-registration-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2e52ad7-bbac-4736-9140-0ce86767eaf2-cni-binary-copy\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-kubelet\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904201 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-multus-certs\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62s5z\" (UniqueName: \"kubernetes.io/projected/89f9861e-d807-40b0-ba0f-3e22c35832bd-kube-api-access-62s5z\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-netns\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904256 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-registration-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.904687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/277edfa4-9e9a-4177-b38d-1bdffa4457ed-kubelet-config\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904340 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904353 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86f4acca-d628-4c62-986a-f66e371b9aa4-hosts-file\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-sys-fs\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-k8s-cni-cncf-io\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-daemon-config\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904433 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86f4acca-d628-4c62-986a-f66e371b9aa4-tmp-dir\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-cni-multus\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904471 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/977bf748-3237-46dd-8171-f882894b111d-serviceca\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904478 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-cni-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djnhn\" (UniqueName: \"kubernetes.io/projected/977bf748-3237-46dd-8171-f882894b111d-kube-api-access-djnhn\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-os-release\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-cni-binary-copy\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904587 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-cnibin\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-cni-bin\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904627 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.905515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/277edfa4-9e9a-4177-b38d-1bdffa4457ed-dbus\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904667 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-netns\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904687 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-conf-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-os-release\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-system-cni-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-socket-dir-parent\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.904876 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-socket-dir-parent\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905160 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.905227 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905269 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2e52ad7-bbac-4736-9140-0ce86767eaf2-cni-binary-copy\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.905285 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:36.405270721 +0000 UTC m=+3.004515913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905310 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-kubelet\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905340 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-etc-selinux\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-cnibin\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.906343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905535 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ce1fb28-3c70-446e-88a6-55447582c3ad-os-release\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905579 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-conf-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-cni-bin\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.905656 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.905711 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:09:36.405695462 +0000 UTC m=+3.004940668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905731 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/277edfa4-9e9a-4177-b38d-1bdffa4457ed-dbus\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ce1fb28-3c70-446e-88a6-55447582c3ad-cni-binary-copy\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905766 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-system-cni-dir\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905791 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-run-k8s-cni-cncf-io\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905836 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-os-release\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905882 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.905769 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a2e52ad7-bbac-4736-9140-0ce86767eaf2-host-var-lib-cni-multus\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.906124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89f9861e-d807-40b0-ba0f-3e22c35832bd-sys-fs\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.906189 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/977bf748-3237-46dd-8171-f882894b111d-serviceca\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.907137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.906236 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a2e52ad7-bbac-4736-9140-0ce86767eaf2-multus-daemon-config\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.914579 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.914282 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:35.914579 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.914303 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:35.914579 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.914316 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:35.914579 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:35.914442 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:36.41442516 +0000 UTC m=+3.013670369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:35.918403 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.918381 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6qw7\" (UniqueName: \"kubernetes.io/projected/79c34749-5c39-4255-8891-20979a6cff44-kube-api-access-d6qw7\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:35.918923 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.918905 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rs7p\" (UniqueName: \"kubernetes.io/projected/9ce1fb28-3c70-446e-88a6-55447582c3ad-kube-api-access-9rs7p\") pod \"multus-additional-cni-plugins-vcb8n\" (UID: \"9ce1fb28-3c70-446e-88a6-55447582c3ad\") " pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:35.924073 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.924017 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62s5z\" (UniqueName: \"kubernetes.io/projected/89f9861e-d807-40b0-ba0f-3e22c35832bd-kube-api-access-62s5z\") pod \"aws-ebs-csi-driver-node-jb2qb\" (UID: \"89f9861e-d807-40b0-ba0f-3e22c35832bd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:35.925974 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.925924 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djnhn\" (UniqueName: \"kubernetes.io/projected/977bf748-3237-46dd-8171-f882894b111d-kube-api-access-djnhn\") pod \"node-ca-2z5z2\" (UID: \"977bf748-3237-46dd-8171-f882894b111d\") " pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:35.927018 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.926999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ggvx\" (UniqueName: \"kubernetes.io/projected/a2e52ad7-bbac-4736-9140-0ce86767eaf2-kube-api-access-4ggvx\") pod \"multus-48gd6\" (UID: \"a2e52ad7-bbac-4736-9140-0ce86767eaf2\") " pod="openshift-multus/multus-48gd6" Jul 24 13:09:35.927841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.927820 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8cpr\" (UniqueName: \"kubernetes.io/projected/86f4acca-d628-4c62-986a-f66e371b9aa4-kube-api-access-b8cpr\") pod \"node-resolver-g5525\" (UID: \"86f4acca-d628-4c62-986a-f66e371b9aa4\") " pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:35.998947 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:35.998903 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:09:36.007816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.007791 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tpbww" Jul 24 13:09:36.018700 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.018683 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7c8qv" Jul 24 13:09:36.025331 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.025313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:36.033030 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.033010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2z5z2" Jul 24 13:09:36.041695 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.041669 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" Jul 24 13:09:36.054471 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.054441 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g5525" Jul 24 13:09:36.063163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.063144 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" Jul 24 13:09:36.067740 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.067724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-48gd6" Jul 24 13:09:36.394606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.394504 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 13:09:36.408181 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.408155 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:36.408293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.408192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:36.408350 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.408314 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:36.408387 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.408375 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.408359113 +0000 UTC m=+4.007604308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:36.408387 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.408314 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:36.408487 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.408414 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.4084051 +0000 UTC m=+4.007650299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:36.509121 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.509086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:36.509317 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.509272 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:36.509317 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.509297 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:36.509317 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.509312 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:36.509485 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.509374 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:37.509360175 +0000 UTC m=+4.108605609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:36.746251 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.746149 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:04:35 +0000 UTC" deadline="2028-03-20 08:42:53.412555626 +0000 UTC" Jul 24 13:09:36.746251 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.746197 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14515h33m16.66636183s" Jul 24 13:09:36.890924 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.890891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:36.891062 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.891001 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:36.891109 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:36.891056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:36.891175 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:36.891158 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:36.933044 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:36.933015 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d35d896_41aa_41e0_ad1e_456449f5a4a8.slice/crio-5ada27392906f91c6618529ce35fd16e4d47f155c5d2642217771bc9a08d784d WatchSource:0}: Error finding container 5ada27392906f91c6618529ce35fd16e4d47f155c5d2642217771bc9a08d784d: Status 404 returned error can't find the container with id 5ada27392906f91c6618529ce35fd16e4d47f155c5d2642217771bc9a08d784d Jul 24 13:09:36.933987 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:36.933755 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24998053_2978_4cb6_8410_6a83cf7418e9.slice/crio-ec7b2e74336379a63b81174b2e54ed491a5f687d17a5178b92f389557afd0348 WatchSource:0}: Error finding container ec7b2e74336379a63b81174b2e54ed491a5f687d17a5178b92f389557afd0348: Status 404 returned error can't find the container with id ec7b2e74336379a63b81174b2e54ed491a5f687d17a5178b92f389557afd0348 Jul 24 13:09:36.940101 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:36.939890 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c393e7a_a2e8_416d_91e8_b18832ac181b.slice/crio-3557161f65f40854629fc6b035c33a8b5f32168504cf4fdaf0d28593e1dc3885 WatchSource:0}: Error finding container 3557161f65f40854629fc6b035c33a8b5f32168504cf4fdaf0d28593e1dc3885: Status 404 returned error can't find the container with id 3557161f65f40854629fc6b035c33a8b5f32168504cf4fdaf0d28593e1dc3885 Jul 24 13:09:36.940880 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:36.940795 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e52ad7_bbac_4736_9140_0ce86767eaf2.slice/crio-b98c94fdf870650e1f81ac3aff5650ae3e7b2ba6ed5a5d20f23484445fb13488 WatchSource:0}: Error finding container b98c94fdf870650e1f81ac3aff5650ae3e7b2ba6ed5a5d20f23484445fb13488: Status 404 returned error can't find the container with id b98c94fdf870650e1f81ac3aff5650ae3e7b2ba6ed5a5d20f23484445fb13488 Jul 24 13:09:36.942227 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:09:36.942207 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod977bf748_3237_46dd_8171_f882894b111d.slice/crio-d36c0bb2258c52a81d2b4fa014d401524cc93ce2bccd9ed0ce3ce3f96dac88b8 WatchSource:0}: Error finding container d36c0bb2258c52a81d2b4fa014d401524cc93ce2bccd9ed0ce3ce3f96dac88b8: Status 404 returned error can't find the container with id d36c0bb2258c52a81d2b4fa014d401524cc93ce2bccd9ed0ce3ce3f96dac88b8 Jul 24 13:09:37.416211 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.415921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:37.416366 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.416080 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:37.416366 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.416247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:37.416366 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.416314 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:39.416281345 +0000 UTC m=+6.015526537 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:37.416541 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.416381 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:37.416541 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.416421 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:09:39.416409782 +0000 UTC m=+6.015654974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:37.519310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.519271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:37.519482 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.519447 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:37.519482 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.519470 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:37.519482 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.519484 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:37.519664 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.519540 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:39.519522326 +0000 UTC m=+6.118767519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:37.747051 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.746930 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 13:04:35 +0000 UTC" deadline="2028-03-04 11:51:03.381795628 +0000 UTC" Jul 24 13:09:37.747051 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.746994 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14134h41m25.634805195s" Jul 24 13:09:37.892148 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.891664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:37.892148 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:37.891800 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:37.906292 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.906261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g5525" event={"ID":"86f4acca-d628-4c62-986a-f66e371b9aa4","Type":"ContainerStarted","Data":"417235a3297680c0fd8b03e615fe2f1fbe7b19d5ed2d92d9f07fc9d34bbace21"} Jul 24 13:09:37.909120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.908991 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerStarted","Data":"ebbb34ca00485b0d955e0bfcd8a661c4a50825339b745a70e522ffd98f48ca49"} Jul 24 13:09:37.912910 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.912831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" event={"ID":"89f9861e-d807-40b0-ba0f-3e22c35832bd","Type":"ContainerStarted","Data":"e061a49681b804a569c1db699bfba276f09b64e00d81e0b3bf744795fb49a76b"} Jul 24 13:09:37.917467 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.917404 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2z5z2" event={"ID":"977bf748-3237-46dd-8171-f882894b111d","Type":"ContainerStarted","Data":"d36c0bb2258c52a81d2b4fa014d401524cc93ce2bccd9ed0ce3ce3f96dac88b8"} Jul 24 13:09:37.924965 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.924901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-48gd6" event={"ID":"a2e52ad7-bbac-4736-9140-0ce86767eaf2","Type":"ContainerStarted","Data":"b98c94fdf870650e1f81ac3aff5650ae3e7b2ba6ed5a5d20f23484445fb13488"} Jul 24 13:09:37.927302 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.927278 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tpbww" event={"ID":"24998053-2978-4cb6-8410-6a83cf7418e9","Type":"ContainerStarted","Data":"ec7b2e74336379a63b81174b2e54ed491a5f687d17a5178b92f389557afd0348"} Jul 24 13:09:37.928902 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.928829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7c8qv" event={"ID":"2d35d896-41aa-41e0-ad1e-456449f5a4a8","Type":"ContainerStarted","Data":"5ada27392906f91c6618529ce35fd16e4d47f155c5d2642217771bc9a08d784d"} Jul 24 13:09:37.931210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.931187 2562 generic.go:356] "Generic (PLEG): container finished" podID="b343ceb436b78e78994571ea8397deb4" containerID="5f5f77eb82edc32fba10293ab9eaf36b3b8b85e5f74c499768e694b8e76eae0f" exitCode=0 Jul 24 13:09:37.931302 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.931260 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" event={"ID":"b343ceb436b78e78994571ea8397deb4","Type":"ContainerDied","Data":"5f5f77eb82edc32fba10293ab9eaf36b3b8b85e5f74c499768e694b8e76eae0f"} Jul 24 13:09:37.955897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.955869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" event={"ID":"be7a3e104963d2d6fa190dbf68a99d50","Type":"ContainerStarted","Data":"3c1f005926f27ee9002b85446b34a03b1733300b65d41153f704d29b853fc0da"} Jul 24 13:09:37.964568 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.964527 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"83d9933826e29fe7084728a83b3ea8bdcfa047d3cbab4dbd60be8a17d875c20d"} Jul 24 13:09:37.976670 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:37.976606 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkb2z" event={"ID":"0c393e7a-a2e8-416d-91e8-b18832ac181b","Type":"ContainerStarted","Data":"3557161f65f40854629fc6b035c33a8b5f32168504cf4fdaf0d28593e1dc3885"} Jul 24 13:09:38.890583 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:38.890535 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:38.891096 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:38.890692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:38.891163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:38.891143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:38.891317 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:38.891265 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:38.984869 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:38.984391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" event={"ID":"b343ceb436b78e78994571ea8397deb4","Type":"ContainerStarted","Data":"4cac91c616f6a602f89b8962cf8fee16903b0790cfdcdc709aaa37e23100df32"} Jul 24 13:09:39.003595 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:39.002872 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-131.ec2.internal" podStartSLOduration=4.002850149 podStartE2EDuration="4.002850149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:37.972739316 +0000 UTC m=+4.571984527" watchObservedRunningTime="2026-07-24 13:09:39.002850149 +0000 UTC m=+5.602095361" Jul 24 13:09:39.435721 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:39.435683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:39.435903 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:39.435752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:39.435991 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.435928 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:39.435991 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.435990 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:43.435970864 +0000 UTC m=+10.035216057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:39.436399 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.436380 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:39.436470 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.436437 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:09:43.436422062 +0000 UTC m=+10.035667257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:39.536581 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:39.536533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:39.536787 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.536770 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:39.536857 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.536798 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:39.536857 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.536812 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:39.536957 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.536867 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:43.536848809 +0000 UTC m=+10.136094003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:39.891223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:39.891153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:39.891653 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:39.891297 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:40.890084 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:40.890051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:40.890231 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:40.890100 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:40.890231 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:40.890184 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:40.890336 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:40.890311 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:41.890158 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:41.890121 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:41.890645 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:41.890238 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:42.890844 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:42.890322 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:42.890844 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:42.890370 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:42.890844 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:42.890463 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:42.890844 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:42.890616 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:43.472347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:43.472313 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:43.472538 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:43.472397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:43.472538 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.472527 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:43.472680 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.472600 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:09:51.472583598 +0000 UTC m=+18.071828793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:43.472979 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.472866 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:43.472979 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.472954 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:51.472930581 +0000 UTC m=+18.072175773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:43.573451 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:43.573415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:43.573676 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.573649 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:43.573676 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.573671 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:43.573791 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.573684 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:43.573791 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.573740 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:09:51.573722964 +0000 UTC m=+18.172968171 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:43.892384 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:43.891307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:43.892384 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:43.891429 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:44.890882 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:44.890387 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:44.890882 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:44.890418 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:44.890882 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:44.890504 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:44.890882 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:44.890659 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:45.890424 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:45.890389 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:45.890900 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:45.890527 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:46.890752 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:46.890716 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:46.890752 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:46.890735 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:46.891233 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:46.890837 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:46.891233 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:46.890935 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:47.890458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:47.890430 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:47.890646 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:47.890571 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:48.890084 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:48.890051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:48.890521 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:48.890061 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:48.890718 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:48.890675 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:48.890826 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:48.890797 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:49.890767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:49.890733 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:49.891192 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:49.890869 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:50.890211 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:50.890185 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:50.890427 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:50.890306 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:50.890427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:50.890362 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:50.890617 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:50.890450 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:51.532114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:51.532082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:51.532530 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:51.532179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:51.532530 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.532256 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:51.532530 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.532270 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:51.532530 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.532327 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:10:07.532313049 +0000 UTC m=+34.131558251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:09:51.532530 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.532346 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:07.532337666 +0000 UTC m=+34.131582870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:09:51.633112 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:51.633065 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:51.633290 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.633243 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:09:51.633290 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.633264 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:09:51.633290 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.633275 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:51.633449 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.633338 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:07.633320575 +0000 UTC m=+34.232565787 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:09:51.890798 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:51.890723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:51.890952 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:51.890857 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:52.890204 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:52.890172 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:52.890671 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:52.890212 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:52.890671 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:52.890297 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:52.890671 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:52.890409 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:53.891073 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:53.891042 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:53.891523 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:53.891165 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:54.890517 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:54.890479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:54.890517 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:54.890508 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:54.890750 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:54.890610 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:54.890801 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:54.890742 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:55.891198 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:55.890816 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:55.891899 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:55.891317 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:56.025813 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.025735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g5525" event={"ID":"86f4acca-d628-4c62-986a-f66e371b9aa4","Type":"ContainerStarted","Data":"3ed45c538e597585db6d0e75c878e25079e0024d10f773daf76dec03e78a5693"} Jul 24 13:09:56.027325 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.027297 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce1fb28-3c70-446e-88a6-55447582c3ad" containerID="218f1352750fea78bf511f4b77ffe8ed36a48e49d2484d47754be2d53a8d7e6e" exitCode=0 Jul 24 13:09:56.027453 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.027370 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerDied","Data":"218f1352750fea78bf511f4b77ffe8ed36a48e49d2484d47754be2d53a8d7e6e"} Jul 24 13:09:56.028858 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.028834 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" event={"ID":"89f9861e-d807-40b0-ba0f-3e22c35832bd","Type":"ContainerStarted","Data":"2de623d9c01a6f0c5ac70470465e339b1d04795c747901cd18fb027b599069a6"} Jul 24 13:09:56.030392 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.030366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2z5z2" event={"ID":"977bf748-3237-46dd-8171-f882894b111d","Type":"ContainerStarted","Data":"eac32fedaa4d06e5f3aaad9b217c69d49e63cd4badb2c7c801fa3213dc0016a9"} Jul 24 13:09:56.031716 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.031693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-48gd6" event={"ID":"a2e52ad7-bbac-4736-9140-0ce86767eaf2","Type":"ContainerStarted","Data":"a28bfe63d754866b08116dec1bddbc127a485e9bee461a0e7e5850e3ef22c154"} Jul 24 13:09:56.033018 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.032995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tpbww" event={"ID":"24998053-2978-4cb6-8410-6a83cf7418e9","Type":"ContainerStarted","Data":"b83c87fa90d9fdeeb8f194de634f06d5b2879fa866c33530837babe000896816"} Jul 24 13:09:56.036171 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.036152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"a487fe3498242d6b7f435fcf2fc0d489282bd6d91173f77b88a0d132f4d11f89"} Jul 24 13:09:56.036252 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.036205 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"e17476d26305a44e740fedc2ef05e5957494022f709dd6d8bb1dbaabcef1bb64"} Jul 24 13:09:56.036252 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.036244 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"6b11151f2d4bb1f45d91e0877ce4bf96839bfea82ef9f10354695281d77a8326"} Jul 24 13:09:56.036344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.036279 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"405e0f2e308890affb5c4b80a6cc5250052ed78d4540321db025e29bd4a3e2e7"} Jul 24 13:09:56.036344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.036308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"cd4442a29a20224bf75eedcf95f27385b2932dd9a611ae44642ff0b9f436558a"} Jul 24 13:09:56.036437 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.036345 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"f2aa35f0a8806bdf1e7b021f95963bd2729ae33f4eb2ae284288efb561eb4b85"} Jul 24 13:09:56.037358 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.037340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkb2z" event={"ID":"0c393e7a-a2e8-416d-91e8-b18832ac181b","Type":"ContainerStarted","Data":"a0d82d925bae76e6068fb16bf4f2a3d5ebd8bcda8bce3fb3b02549ca8d1ff7c5"} Jul 24 13:09:56.041295 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.041263 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-131.ec2.internal" podStartSLOduration=21.041238251 podStartE2EDuration="21.041238251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:39.003533116 +0000 UTC m=+5.602778344" watchObservedRunningTime="2026-07-24 13:09:56.041238251 +0000 UTC m=+22.640483462" Jul 24 13:09:56.065429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.065392 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-g5525" podStartSLOduration=3.810321754 podStartE2EDuration="22.065381261s" totalImagesPullingTime="18.255059507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:56.041026331 +0000 UTC m=+22.640271543" watchObservedRunningTime="2026-07-24 13:09:56.065381261 +0000 UTC m=+22.664626527" Jul 24 13:09:56.084561 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.084501 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-48gd6" podStartSLOduration=3.770031015 podStartE2EDuration="22.084484306s" totalImagesPullingTime="18.314453291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:56.08437142 +0000 UTC m=+22.683616632" watchObservedRunningTime="2026-07-24 13:09:56.084484306 +0000 UTC m=+22.683729544" Jul 24 13:09:56.099618 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.099581 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2z5z2" podStartSLOduration=3.844049477 podStartE2EDuration="22.09957238s" totalImagesPullingTime="18.255522903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:56.099064623 +0000 UTC m=+22.698309838" watchObservedRunningTime="2026-07-24 13:09:56.09957238 +0000 UTC m=+22.698817588" Jul 24 13:09:56.115414 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.115367 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tpbww" podStartSLOduration=4.845510158 podStartE2EDuration="23.115355003s" totalImagesPullingTime="18.269844845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:56.115199716 +0000 UTC m=+22.714444930" watchObservedRunningTime="2026-07-24 13:09:56.115355003 +0000 UTC m=+22.714600214" Jul 24 13:09:56.130215 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.130169 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xkb2z" podStartSLOduration=12.786510506 podStartE2EDuration="23.130153468s" totalImagesPullingTime="10.343642962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:56.129700023 +0000 UTC m=+22.728945234" watchObservedRunningTime="2026-07-24 13:09:56.130153468 +0000 UTC m=+22.729398684" Jul 24 13:09:56.583344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.583318 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 13:09:56.773179 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.773001 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T13:09:56.583333103Z","UUID":"b19ec18a-620b-4f22-adaa-ccf2f1cb9186","Handler":null,"Name":"","Endpoint":""} Jul 24 13:09:56.774809 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.774785 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 13:09:56.774937 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.774833 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 13:09:56.890799 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.890768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:56.890949 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:56.890885 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:56.890949 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:56.890931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:56.891049 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:56.891013 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:57.042171 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:57.042081 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" event={"ID":"89f9861e-d807-40b0-ba0f-3e22c35832bd","Type":"ContainerStarted","Data":"8da655ff5c83e52ed3eaa0c456e4b0597cb68c0b29921958feea8165cf35fb0d"} Jul 24 13:09:57.043769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:57.043739 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7c8qv" event={"ID":"2d35d896-41aa-41e0-ad1e-456449f5a4a8","Type":"ContainerStarted","Data":"344052fa8d971af08fdfe7ea5dbe1ed06b7311b1a11e1ff7047f46551e8a0622"} Jul 24 13:09:57.057600 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:57.057562 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7c8qv" podStartSLOduration=4.788871745 podStartE2EDuration="23.057532893s" totalImagesPullingTime="18.268661148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:57.057338566 +0000 UTC m=+23.656583777" watchObservedRunningTime="2026-07-24 13:09:57.057532893 +0000 UTC m=+23.656778103" Jul 24 13:09:57.527496 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:57.527464 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:57.528125 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:57.528100 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:57.890338 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:57.890314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:57.890473 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:57.890447 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:09:58.050425 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.050143 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"9d6ec7ff1253f34f7c523fd703a0db6502b52f70dfcc1ce23cf9d7990686b819"} Jul 24 13:09:58.052458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.052385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" event={"ID":"89f9861e-d807-40b0-ba0f-3e22c35832bd","Type":"ContainerStarted","Data":"b69086fd17a414a8a1bb81d6dee84ecf4f11c8cd64f973c43f39b3200e8ebc60"} Jul 24 13:09:58.052882 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.052858 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:58.053154 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.053119 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xkb2z" Jul 24 13:09:58.070283 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.070238 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jb2qb" podStartSLOduration=3.549359575 podStartE2EDuration="24.070220842s" totalImagesPullingTime="20.520861267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:58.069712722 +0000 UTC m=+24.668957932" watchObservedRunningTime="2026-07-24 13:09:58.070220842 +0000 UTC m=+24.669466054" Jul 24 13:09:58.890338 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.890311 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:09:58.890338 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:58.890327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:09:58.890684 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:58.890430 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:09:58.891057 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:58.890907 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:09:59.890681 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:09:59.890646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:09:59.891167 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:09:59.890766 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:10:00.890595 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:00.890396 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:00.890727 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:00.890465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:00.890727 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:00.890686 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:10:00.891227 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:00.890746 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:10:01.060454 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.060369 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce1fb28-3c70-446e-88a6-55447582c3ad" containerID="d59673f334a9780bcb518954d68076db1105b7faf214f1df16d8235b50dfbd92" exitCode=0 Jul 24 13:10:01.060632 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.060462 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerDied","Data":"d59673f334a9780bcb518954d68076db1105b7faf214f1df16d8235b50dfbd92"} Jul 24 13:10:01.064120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.064088 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" event={"ID":"a5a5d75c-2df9-41a5-8601-bdc2d2d95ed3","Type":"ContainerStarted","Data":"ff42d56a965fd7ea1b6c18b3dd4ee59e3790a049abbd54c69caa2faf7a54f37d"} Jul 24 13:10:01.064470 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.064454 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:10:01.064579 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.064525 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:10:01.064644 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.064591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:10:01.079618 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.079594 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:10:01.079838 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.079819 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:10:01.103522 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.103482 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" podStartSLOduration=9.784596095 podStartE2EDuration="28.103468656s" totalImagesPullingTime="18.318872561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:01.102567687 +0000 UTC m=+27.701812899" watchObservedRunningTime="2026-07-24 13:10:01.103468656 +0000 UTC m=+27.702713867" Jul 24 13:10:01.890609 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:01.890583 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:01.890744 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:01.890711 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:10:02.890419 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:02.890382 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:02.890419 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:02.890416 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:02.890664 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:02.890496 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:10:02.890664 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:02.890584 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:10:03.069998 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:03.069970 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce1fb28-3c70-446e-88a6-55447582c3ad" containerID="8d90ba0a06f18284331ca4df19fd3965fba7c1176a8949a950177b5e6d3a672f" exitCode=0 Jul 24 13:10:03.070340 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:03.070056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerDied","Data":"8d90ba0a06f18284331ca4df19fd3965fba7c1176a8949a950177b5e6d3a672f"} Jul 24 13:10:03.890619 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:03.890585 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:03.890796 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:03.890692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:10:04.890300 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:04.890272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:04.890686 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:04.890369 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:10:04.890686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:04.890397 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:04.890686 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:04.890452 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:10:05.077249 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:05.077219 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce1fb28-3c70-446e-88a6-55447582c3ad" containerID="206082e88af15ddcc89db99c4d99c614b231db9cfdad994aa2d6d153c0fc67d0" exitCode=0 Jul 24 13:10:05.077405 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:05.077278 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerDied","Data":"206082e88af15ddcc89db99c4d99c614b231db9cfdad994aa2d6d153c0fc67d0"} Jul 24 13:10:05.890587 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:05.890558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:05.891039 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:05.890702 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:10:06.890408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:06.890378 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:06.890408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:06.890416 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:06.890665 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:06.890503 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:10:06.890981 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:06.890662 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:10:07.559865 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.559597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:07.560052 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.559919 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:07.560118 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.560042 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:10:07.560118 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.560110 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 13:10:07.560214 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.560132 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs podName:79c34749-5c39-4255-8891-20979a6cff44 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:39.560111346 +0000 UTC m=+66.159356551 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs") pod "network-metrics-daemon-7tttb" (UID: "79c34749-5c39-4255-8891-20979a6cff44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 13:10:07.560214 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.560168 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret podName:277edfa4-9e9a-4177-b38d-1bdffa4457ed nodeName:}" failed. No retries permitted until 2026-07-24 13:10:39.560148685 +0000 UTC m=+66.159393875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret") pod "global-pull-secret-syncer-dv8tx" (UID: "277edfa4-9e9a-4177-b38d-1bdffa4457ed") : object "kube-system"/"original-pull-secret" not registered Jul 24 13:10:07.660462 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.660428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:07.660648 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.660576 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 13:10:07.660648 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.660592 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 13:10:07.660648 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.660601 2562 projected.go:196] Error preparing data for projected volume kube-api-access-gfzq7 for pod openshift-network-diagnostics/network-check-target-8ncd6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:10:07.660806 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.660653 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7 podName:7eb52081-e188-4f54-9448-cafd8edb1c96 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:39.660637744 +0000 UTC m=+66.259882954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gfzq7" (UniqueName: "kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7") pod "network-check-target-8ncd6" (UID: "7eb52081-e188-4f54-9448-cafd8edb1c96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 13:10:07.849202 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.848785 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7tttb"] Jul 24 13:10:07.849202 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.848956 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:07.849202 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.849086 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:10:07.849619 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.849594 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dv8tx"] Jul 24 13:10:07.849748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.849684 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:07.849837 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.849770 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:10:07.851057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.851031 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ncd6"] Jul 24 13:10:07.851216 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:07.851134 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:07.851306 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:07.851225 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:10:09.890271 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:09.890194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:09.890271 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:09.890215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:09.890932 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:09.890194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:09.890932 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:09.890289 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8tx" podUID="277edfa4-9e9a-4177-b38d-1bdffa4457ed" Jul 24 13:10:09.890932 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:09.890402 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tttb" podUID="79c34749-5c39-4255-8891-20979a6cff44" Jul 24 13:10:09.890932 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:09.890450 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8ncd6" podUID="7eb52081-e188-4f54-9448-cafd8edb1c96" Jul 24 13:10:10.093077 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.093050 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce1fb28-3c70-446e-88a6-55447582c3ad" containerID="fcc97288094b04b3e46d1754c5c875d857135ee1b9bd250c291a67ea5b4b4740" exitCode=0 Jul 24 13:10:10.093221 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.093107 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerDied","Data":"fcc97288094b04b3e46d1754c5c875d857135ee1b9bd250c291a67ea5b4b4740"} Jul 24 13:10:10.756917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.756701 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-131.ec2.internal" event="NodeReady" Jul 24 13:10:10.757081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.757066 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 13:10:10.799187 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.799135 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dd4f6b6c8-zhkv2"] Jul 24 13:10:10.802276 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.802260 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.805693 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.805658 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 13:10:10.805877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.805856 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 13:10:10.805966 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.805911 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kvvpn\"" Jul 24 13:10:10.806607 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.806033 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 13:10:10.816123 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.816106 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 13:10:10.832715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.832690 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xk4j5"] Jul 24 13:10:10.838375 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.838355 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sn7tj"] Jul 24 13:10:10.838492 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.838476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:10.841221 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.841206 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 13:10:10.841355 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.841339 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 13:10:10.841421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.841356 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 13:10:10.841656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.841641 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-t8pk6\"" Jul 24 13:10:10.842177 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.842161 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd4f6b6c8-zhkv2"] Jul 24 13:10:10.842244 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.842235 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:10.844999 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.844980 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 13:10:10.845093 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.845080 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 13:10:10.845159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.845112 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 13:10:10.845159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.845132 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 13:10:10.845159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.845137 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xhz7z\"" Jul 24 13:10:10.849599 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.849576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sn7tj"] Jul 24 13:10:10.849926 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.849907 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xk4j5"] Jul 24 13:10:10.883572 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-registry-tls\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883681 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883630 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87c96c12-0d56-4d4c-b3b5-623198e1a45d-registry-certificates\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883681 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvj2\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-kube-api-access-djvj2\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883775 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87c96c12-0d56-4d4c-b3b5-623198e1a45d-image-registry-private-configuration\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883811 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87c96c12-0d56-4d4c-b3b5-623198e1a45d-ca-trust-extracted\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883811 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883797 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87c96c12-0d56-4d4c-b3b5-623198e1a45d-trusted-ca\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87c96c12-0d56-4d4c-b3b5-623198e1a45d-installation-pull-secrets\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.883877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.883850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-bound-sa-token\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.911818 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.911793 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m4pgm"] Jul 24 13:10:10.914822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.914806 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:10.916923 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.916905 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n54jn\"" Jul 24 13:10:10.917005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.916905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 13:10:10.917089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.917068 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 13:10:10.923991 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.923971 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4pgm"] Jul 24 13:10:10.984926 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.984830 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2-cert\") pod \"ingress-canary-xk4j5\" (UID: \"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2\") " pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:10.984926 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.984902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-registry-tls\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985110 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.984932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb8xm\" (UniqueName: \"kubernetes.io/projected/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-kube-api-access-vb8xm\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:10.985110 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.984963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-crio-socket\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:10.985110 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.984992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sdjb\" (UniqueName: \"kubernetes.io/projected/2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2-kube-api-access-6sdjb\") pod \"ingress-canary-xk4j5\" (UID: \"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2\") " pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:10.985110 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87c96c12-0d56-4d4c-b3b5-623198e1a45d-registry-certificates\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985110 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985053 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-data-volume\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:10.985110 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvj2\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-kube-api-access-djvj2\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985121 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87c96c12-0d56-4d4c-b3b5-623198e1a45d-image-registry-private-configuration\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87c96c12-0d56-4d4c-b3b5-623198e1a45d-ca-trust-extracted\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87c96c12-0d56-4d4c-b3b5-623198e1a45d-trusted-ca\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87c96c12-0d56-4d4c-b3b5-623198e1a45d-installation-pull-secrets\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-bound-sa-token\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985362 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:10.985702 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985518 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/87c96c12-0d56-4d4c-b3b5-623198e1a45d-ca-trust-extracted\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.985907 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.985888 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/87c96c12-0d56-4d4c-b3b5-623198e1a45d-registry-certificates\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.986337 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.986189 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/87c96c12-0d56-4d4c-b3b5-623198e1a45d-trusted-ca\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.989318 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.989301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/87c96c12-0d56-4d4c-b3b5-623198e1a45d-installation-pull-secrets\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.989410 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.989312 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/87c96c12-0d56-4d4c-b3b5-623198e1a45d-image-registry-private-configuration\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.989410 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.989326 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-registry-tls\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.992645 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.992623 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-bound-sa-token\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:10.992936 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:10.992920 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvj2\" (UniqueName: \"kubernetes.io/projected/87c96c12-0d56-4d4c-b3b5-623198e1a45d-kube-api-access-djvj2\") pod \"image-registry-dd4f6b6c8-zhkv2\" (UID: \"87c96c12-0d56-4d4c-b3b5-623198e1a45d\") " pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:11.085594 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb8xm\" (UniqueName: \"kubernetes.io/projected/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-kube-api-access-vb8xm\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.085749 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085600 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-crio-socket\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.085749 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sdjb\" (UniqueName: \"kubernetes.io/projected/2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2-kube-api-access-6sdjb\") pod \"ingress-canary-xk4j5\" (UID: \"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2\") " pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:11.085857 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-data-volume\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.085857 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085763 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-crio-socket\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.085857 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40a02687-5781-4843-8152-083a322887c0-metrics-tls\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.085857 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.085857 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40a02687-5781-4843-8152-083a322887c0-config-volume\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.086103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a02687-5781-4843-8152-083a322887c0-tmp-dir\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.086103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085911 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4ndc\" (UniqueName: \"kubernetes.io/projected/40a02687-5781-4843-8152-083a322887c0-kube-api-access-m4ndc\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.086103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.085977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.086103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.086034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2-cert\") pod \"ingress-canary-xk4j5\" (UID: \"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2\") " pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:11.086103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.086078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-data-volume\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.086336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.086320 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.088030 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.088011 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.088137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.088119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2-cert\") pod \"ingress-canary-xk4j5\" (UID: \"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2\") " pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:11.093503 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.093480 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb8xm\" (UniqueName: \"kubernetes.io/projected/7586cdf8-d5f7-4c83-81d5-73d6de053f9b-kube-api-access-vb8xm\") pod \"insights-runtime-extractor-sn7tj\" (UID: \"7586cdf8-d5f7-4c83-81d5-73d6de053f9b\") " pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.093969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.093951 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sdjb\" (UniqueName: \"kubernetes.io/projected/2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2-kube-api-access-6sdjb\") pod \"ingress-canary-xk4j5\" (UID: \"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2\") " pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:11.097977 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.097951 2562 generic.go:356] "Generic (PLEG): container finished" podID="9ce1fb28-3c70-446e-88a6-55447582c3ad" containerID="2f6380d942fd4e3dd6c8c1eaee0bf6eb978d9685dc1767e868c86a1a49c9d94b" exitCode=0 Jul 24 13:10:11.098057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.097988 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerDied","Data":"2f6380d942fd4e3dd6c8c1eaee0bf6eb978d9685dc1767e868c86a1a49c9d94b"} Jul 24 13:10:11.111388 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.111368 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:11.148200 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.148156 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xk4j5" Jul 24 13:10:11.153880 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.153856 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sn7tj" Jul 24 13:10:11.186490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.186463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40a02687-5781-4843-8152-083a322887c0-metrics-tls\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.186611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.186527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40a02687-5781-4843-8152-083a322887c0-config-volume\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.186611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.186578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a02687-5781-4843-8152-083a322887c0-tmp-dir\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.186611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.186603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4ndc\" (UniqueName: \"kubernetes.io/projected/40a02687-5781-4843-8152-083a322887c0-kube-api-access-m4ndc\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.187269 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.186956 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40a02687-5781-4843-8152-083a322887c0-tmp-dir\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.187269 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.187158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/40a02687-5781-4843-8152-083a322887c0-config-volume\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.189850 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.189825 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/40a02687-5781-4843-8152-083a322887c0-metrics-tls\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.194698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.194679 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4ndc\" (UniqueName: \"kubernetes.io/projected/40a02687-5781-4843-8152-083a322887c0-kube-api-access-m4ndc\") pod \"dns-default-m4pgm\" (UID: \"40a02687-5781-4843-8152-083a322887c0\") " pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.223913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.223885 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:11.349810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.349470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xk4j5"] Jul 24 13:10:11.363052 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:11.363021 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d14cd9d_a3ef_46c4_abd7_b334b0f0afb2.slice/crio-174adc8577d1fa6c4d835eabfc025a0775848326cfb53fc7ce86c5f1c22c1585 WatchSource:0}: Error finding container 174adc8577d1fa6c4d835eabfc025a0775848326cfb53fc7ce86c5f1c22c1585: Status 404 returned error can't find the container with id 174adc8577d1fa6c4d835eabfc025a0775848326cfb53fc7ce86c5f1c22c1585 Jul 24 13:10:11.369672 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.369650 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dd4f6b6c8-zhkv2"] Jul 24 13:10:11.372642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.372541 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sn7tj"] Jul 24 13:10:11.373214 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:11.373184 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c96c12_0d56_4d4c_b3b5_623198e1a45d.slice/crio-668e33f564eee0c9a65cd16d35261a457fce9ab23711327c90e691c63c900450 WatchSource:0}: Error finding container 668e33f564eee0c9a65cd16d35261a457fce9ab23711327c90e691c63c900450: Status 404 returned error can't find the container with id 668e33f564eee0c9a65cd16d35261a457fce9ab23711327c90e691c63c900450 Jul 24 13:10:11.376045 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:11.376021 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7586cdf8_d5f7_4c83_81d5_73d6de053f9b.slice/crio-f3619373e4fb8a138a47a3bacfcdb187668a4a3a3f20d69036dd88abebb1aba6 WatchSource:0}: Error finding container f3619373e4fb8a138a47a3bacfcdb187668a4a3a3f20d69036dd88abebb1aba6: Status 404 returned error can't find the container with id f3619373e4fb8a138a47a3bacfcdb187668a4a3a3f20d69036dd88abebb1aba6 Jul 24 13:10:11.408146 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.408126 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4pgm"] Jul 24 13:10:11.411455 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:11.411429 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a02687_5781_4843_8152_083a322887c0.slice/crio-08c6c195ee7f54a493fbd3f59a07d27c61d370a27b7890ea9bb3ccdbc4112d77 WatchSource:0}: Error finding container 08c6c195ee7f54a493fbd3f59a07d27c61d370a27b7890ea9bb3ccdbc4112d77: Status 404 returned error can't find the container with id 08c6c195ee7f54a493fbd3f59a07d27c61d370a27b7890ea9bb3ccdbc4112d77 Jul 24 13:10:11.895024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.894996 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:11.895336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.895321 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:11.895741 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.895572 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:11.897751 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.897727 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:10:11.898204 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.898139 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:10:11.898624 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.898564 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rh25q\"" Jul 24 13:10:11.898721 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.898623 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dqwnd\"" Jul 24 13:10:11.898844 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.898828 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:10:11.899042 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:11.899025 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:10:12.102622 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.102539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xk4j5" event={"ID":"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2","Type":"ContainerStarted","Data":"174adc8577d1fa6c4d835eabfc025a0775848326cfb53fc7ce86c5f1c22c1585"} Jul 24 13:10:12.104158 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.104051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4pgm" event={"ID":"40a02687-5781-4843-8152-083a322887c0","Type":"ContainerStarted","Data":"08c6c195ee7f54a493fbd3f59a07d27c61d370a27b7890ea9bb3ccdbc4112d77"} Jul 24 13:10:12.106952 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.106883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" event={"ID":"87c96c12-0d56-4d4c-b3b5-623198e1a45d","Type":"ContainerStarted","Data":"27481fa4e3023af1f864b46d0e69ed72310d0e8be395a8f50b321fd83703d525"} Jul 24 13:10:12.106952 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.106928 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" event={"ID":"87c96c12-0d56-4d4c-b3b5-623198e1a45d","Type":"ContainerStarted","Data":"668e33f564eee0c9a65cd16d35261a457fce9ab23711327c90e691c63c900450"} Jul 24 13:10:12.107170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.107154 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:12.113751 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.113729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" event={"ID":"9ce1fb28-3c70-446e-88a6-55447582c3ad","Type":"ContainerStarted","Data":"266ddca8a339743b5949876454d6edb0e15f587612026d865ac5a48982d3ccfe"} Jul 24 13:10:12.115643 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.115620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sn7tj" event={"ID":"7586cdf8-d5f7-4c83-81d5-73d6de053f9b","Type":"ContainerStarted","Data":"f3619373e4fb8a138a47a3bacfcdb187668a4a3a3f20d69036dd88abebb1aba6"} Jul 24 13:10:12.129168 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.129024 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" podStartSLOduration=6.129006196 podStartE2EDuration="6.129006196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:12.127720868 +0000 UTC m=+38.726966080" watchObservedRunningTime="2026-07-24 13:10:12.129006196 +0000 UTC m=+38.728251405" Jul 24 13:10:12.148292 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:12.148208 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vcb8n" podStartSLOduration=9.122560209 podStartE2EDuration="38.148191397s" totalImagesPullingTime="29.025631188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:12.147269366 +0000 UTC m=+38.746514578" watchObservedRunningTime="2026-07-24 13:10:12.148191397 +0000 UTC m=+38.747436609" Jul 24 13:10:13.944461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.944404 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-df5db56cb-kk4sx"] Jul 24 13:10:13.958701 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.958679 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df5db56cb-kk4sx"] Jul 24 13:10:13.958817 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.958772 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:13.961175 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.961151 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 13:10:13.961919 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.961896 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 13:10:13.962105 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.961969 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 13:10:13.962364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.962225 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 13:10:13.962364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.962262 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 13:10:13.962364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.962276 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 13:10:13.962364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.962284 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z656h\"" Jul 24 13:10:13.962364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.962284 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 13:10:13.965851 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:13.965834 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 13:10:14.110022 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.109990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-serving-cert\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.110190 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.110039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-config\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.110190 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.110060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-trusted-ca-bundle\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.110190 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.110135 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-oauth-config\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.110190 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.110158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-oauth-serving-cert\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.110336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.110252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-service-ca\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.110336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.110281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzcsl\" (UniqueName: \"kubernetes.io/projected/0ebfd84e-f4dc-4314-b9e7-4356005a892e-kube-api-access-gzcsl\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.210780 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210697 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-serving-cert\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.210780 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-config\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.210780 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210768 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-trusted-ca-bundle\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211068 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-oauth-config\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211068 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-oauth-serving-cert\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211068 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-service-ca\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211068 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.210913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzcsl\" (UniqueName: \"kubernetes.io/projected/0ebfd84e-f4dc-4314-b9e7-4356005a892e-kube-api-access-gzcsl\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211651 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.211618 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-config\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211760 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.211721 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-oauth-serving-cert\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211817 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.211781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-service-ca\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.211872 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.211848 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-trusted-ca-bundle\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.214966 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.214939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-serving-cert\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.215140 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.215124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-oauth-config\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.219438 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.219415 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzcsl\" (UniqueName: \"kubernetes.io/projected/0ebfd84e-f4dc-4314-b9e7-4356005a892e-kube-api-access-gzcsl\") pod \"console-df5db56cb-kk4sx\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:14.268383 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:14.268352 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:18.362823 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:18.362791 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df5db56cb-kk4sx"] Jul 24 13:10:18.365893 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:18.365867 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebfd84e_f4dc_4314_b9e7_4356005a892e.slice/crio-a5339fedec9cb47c2d57d32263ad149f8e56f027ca18537a54fac0b4e3fd4319 WatchSource:0}: Error finding container a5339fedec9cb47c2d57d32263ad149f8e56f027ca18537a54fac0b4e3fd4319: Status 404 returned error can't find the container with id a5339fedec9cb47c2d57d32263ad149f8e56f027ca18537a54fac0b4e3fd4319 Jul 24 13:10:19.137153 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.137115 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4pgm" event={"ID":"40a02687-5781-4843-8152-083a322887c0","Type":"ContainerStarted","Data":"ba9aaf1615fdc214bcf8d847dca90fa769bc48cf71d35868da85fb5b9f0ae93a"} Jul 24 13:10:19.137343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.137173 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4pgm" event={"ID":"40a02687-5781-4843-8152-083a322887c0","Type":"ContainerStarted","Data":"38359201c0c165efac71e9c1e9e37b2d014fb6a73ded354ade483327632cbf23"} Jul 24 13:10:19.137343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.137200 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:19.138484 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.138461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sn7tj" event={"ID":"7586cdf8-d5f7-4c83-81d5-73d6de053f9b","Type":"ContainerStarted","Data":"c7ebc56daf32812313253950e0bdd7029fe748649f1086a6efbfb41a41208877"} Jul 24 13:10:19.139843 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.139816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xk4j5" event={"ID":"2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2","Type":"ContainerStarted","Data":"66c7b474d9a0f57e4bd21bcf27b497ecc165ad27e31d50a3a9716ad064c1793d"} Jul 24 13:10:19.140913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.140893 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df5db56cb-kk4sx" event={"ID":"0ebfd84e-f4dc-4314-b9e7-4356005a892e","Type":"ContainerStarted","Data":"a5339fedec9cb47c2d57d32263ad149f8e56f027ca18537a54fac0b4e3fd4319"} Jul 24 13:10:19.153420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.153383 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m4pgm" podStartSLOduration=2.321727711 podStartE2EDuration="9.153371185s" totalImagesPullingTime="6.831643474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:19.152952485 +0000 UTC m=+45.752197696" watchObservedRunningTime="2026-07-24 13:10:19.153371185 +0000 UTC m=+45.752616374" Jul 24 13:10:19.167924 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:19.167883 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xk4j5" podStartSLOduration=2.288122435 podStartE2EDuration="9.167872201s" totalImagesPullingTime="6.879749766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:19.167227367 +0000 UTC m=+45.766472578" watchObservedRunningTime="2026-07-24 13:10:19.167872201 +0000 UTC m=+45.767117412" Jul 24 13:10:25.160770 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:25.160684 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sn7tj" event={"ID":"7586cdf8-d5f7-4c83-81d5-73d6de053f9b","Type":"ContainerStarted","Data":"86a764fe40440d778027d75fb54be4bc087c6aaee656ff0e8560b335bca4a916"} Jul 24 13:10:25.162051 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:25.162031 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df5db56cb-kk4sx" event={"ID":"0ebfd84e-f4dc-4314-b9e7-4356005a892e","Type":"ContainerStarted","Data":"31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883"} Jul 24 13:10:25.178421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:25.178381 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-df5db56cb-kk4sx" podStartSLOduration=6.113784622 podStartE2EDuration="12.1783681s" totalImagesPullingTime="6.064583478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:25.177226819 +0000 UTC m=+51.776472027" watchObservedRunningTime="2026-07-24 13:10:25.1783681 +0000 UTC m=+51.777613310" Jul 24 13:10:29.146982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:29.146955 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m4pgm" Jul 24 13:10:29.176800 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:29.176768 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sn7tj" event={"ID":"7586cdf8-d5f7-4c83-81d5-73d6de053f9b","Type":"ContainerStarted","Data":"b3d1da076ac516a32b936174308cc977c601c4c2c2614821bd5171cc4fc046b4"} Jul 24 13:10:29.193799 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:29.193757 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sn7tj" podStartSLOduration=2.4243754539999998 podStartE2EDuration="19.19374199s" totalImagesPullingTime="16.769366536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:10:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:29.192732049 +0000 UTC m=+55.791977257" watchObservedRunningTime="2026-07-24 13:10:29.19374199 +0000 UTC m=+55.792987224" Jul 24 13:10:31.115948 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:31.115915 2562 patch_prober.go:28] interesting pod/image-registry-dd4f6b6c8-zhkv2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 13:10:31.116296 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:31.115978 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" podUID="87c96c12-0d56-4d4c-b3b5-623198e1a45d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:10:33.080634 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:33.080604 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dl6cj" Jul 24 13:10:33.124434 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:33.124411 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dd4f6b6c8-zhkv2" Jul 24 13:10:34.269119 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:34.269078 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:34.269621 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:34.269147 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:34.273707 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:34.273687 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:35.196372 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:35.196345 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:10:39.230169 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.230138 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5"] Jul 24 13:10:39.232768 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.232752 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.235174 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.235154 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 13:10:39.235391 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.235360 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 24 13:10:39.235391 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.235367 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 13:10:39.235538 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.235423 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 24 13:10:39.236056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.236041 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 13:10:39.236128 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.236113 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qbqtm\"" Jul 24 13:10:39.245614 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.245596 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h55pl"] Jul 24 13:10:39.247302 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.247284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.247779 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.247760 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm"] Jul 24 13:10:39.249234 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.249217 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 13:10:39.249478 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.249460 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 13:10:39.249604 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.249587 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8tk55\"" Jul 24 13:10:39.249667 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.249607 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 13:10:39.249890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.249875 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5"] Jul 24 13:10:39.249983 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.249970 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.252529 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.252502 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 24 13:10:39.252722 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.252703 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ls59m\"" Jul 24 13:10:39.253043 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.253027 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 24 13:10:39.253140 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.253078 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 24 13:10:39.263355 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.263337 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm"] Jul 24 13:10:39.302451 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-textfile\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302570 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302457 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25244\" (UniqueName: \"kubernetes.io/projected/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-kube-api-access-25244\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302570 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.302671 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302671 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495efb19-5d76-45a8-9f29-3dfca443e378-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.302671 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v49q\" (UniqueName: \"kubernetes.io/projected/14a76ed7-c3d3-4843-9f7b-3f0619399d57-kube-api-access-2v49q\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.302671 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302665 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-sys\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302744 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-wtmp\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-656fv\" (UniqueName: \"kubernetes.io/projected/495efb19-5d76-45a8-9f29-3dfca443e378-kube-api-access-656fv\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.302841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-root\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.302841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-tls\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-metrics-client-ca\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302938 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.302979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/495efb19-5d76-45a8-9f29-3dfca443e378-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.303009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14a76ed7-c3d3-4843-9f7b-3f0619399d57-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.303047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.303081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.303072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.403982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.403952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-sys\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.403982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.403983 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-wtmp\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.403999 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-656fv\" (UniqueName: \"kubernetes.io/projected/495efb19-5d76-45a8-9f29-3dfca443e378-kube-api-access-656fv\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-root\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-tls\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-metrics-client-ca\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404076 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-sys\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404122 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-wtmp\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404124 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-root\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404148 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/495efb19-5d76-45a8-9f29-3dfca443e378-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404271 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14a76ed7-c3d3-4843-9f7b-3f0619399d57-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-textfile\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404380 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25244\" (UniqueName: \"kubernetes.io/projected/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-kube-api-access-25244\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404408 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404465 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495efb19-5d76-45a8-9f29-3dfca443e378-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404527 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v49q\" (UniqueName: \"kubernetes.io/projected/14a76ed7-c3d3-4843-9f7b-3f0619399d57-kube-api-access-2v49q\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404573 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404630 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/495efb19-5d76-45a8-9f29-3dfca443e378-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-textfile\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:39.404833 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.404843 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-metrics-client-ca\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.404994 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:39.404897 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-tls podName:14a76ed7-c3d3-4843-9f7b-3f0619399d57 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:39.904879605 +0000 UTC m=+66.504124797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-pzpv5" (UID: "14a76ed7-c3d3-4843-9f7b-3f0619399d57") : secret "openshift-state-metrics-tls" not found Jul 24 13:10:39.405416 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:39.405015 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 24 13:10:39.405416 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:39.405065 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls podName:495efb19-5d76-45a8-9f29-3dfca443e378 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:39.905049047 +0000 UTC m=+66.504294249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-m6brm" (UID: "495efb19-5d76-45a8-9f29-3dfca443e378") : secret "kube-state-metrics-tls" not found Jul 24 13:10:39.405416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.405101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-accelerators-collector-config\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.405648 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.405623 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.405754 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.405675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/495efb19-5d76-45a8-9f29-3dfca443e378-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.405877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.405859 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14a76ed7-c3d3-4843-9f7b-3f0619399d57-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.406733 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.406710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.406900 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.406882 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.407090 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.407071 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.407176 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.407161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-node-exporter-tls\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.413955 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.413936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-656fv\" (UniqueName: \"kubernetes.io/projected/495efb19-5d76-45a8-9f29-3dfca443e378-kube-api-access-656fv\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.414131 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.414112 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v49q\" (UniqueName: \"kubernetes.io/projected/14a76ed7-c3d3-4843-9f7b-3f0619399d57-kube-api-access-2v49q\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.414418 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.414400 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25244\" (UniqueName: \"kubernetes.io/projected/e09e824f-b26a-44b8-aff1-c7f2caf46b9a-kube-api-access-25244\") pod \"node-exporter-h55pl\" (UID: \"e09e824f-b26a-44b8-aff1-c7f2caf46b9a\") " pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.557155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.557075 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h55pl" Jul 24 13:10:39.566560 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:39.566523 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09e824f_b26a_44b8_aff1_c7f2caf46b9a.slice/crio-c9de06cd8ea1b046700df090da53265f0bec5e64eaad0a9ce27dfc0348cd4de3 WatchSource:0}: Error finding container c9de06cd8ea1b046700df090da53265f0bec5e64eaad0a9ce27dfc0348cd4de3: Status 404 returned error can't find the container with id c9de06cd8ea1b046700df090da53265f0bec5e64eaad0a9ce27dfc0348cd4de3 Jul 24 13:10:39.606808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.606787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:39.606890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.606830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:39.608963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.608945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 13:10:39.609046 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.609012 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 13:10:39.619231 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.619208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/277edfa4-9e9a-4177-b38d-1bdffa4457ed-original-pull-secret\") pod \"global-pull-secret-syncer-dv8tx\" (UID: \"277edfa4-9e9a-4177-b38d-1bdffa4457ed\") " pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:39.619315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.619230 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c34749-5c39-4255-8891-20979a6cff44-metrics-certs\") pod \"network-metrics-daemon-7tttb\" (UID: \"79c34749-5c39-4255-8891-20979a6cff44\") " pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:39.708174 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.708147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:39.710159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.710140 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 13:10:39.720852 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.720835 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 13:10:39.731900 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.731878 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfzq7\" (UniqueName: \"kubernetes.io/projected/7eb52081-e188-4f54-9448-cafd8edb1c96-kube-api-access-gfzq7\") pod \"network-check-target-8ncd6\" (UID: \"7eb52081-e188-4f54-9448-cafd8edb1c96\") " pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:39.809265 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.809204 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rh25q\"" Jul 24 13:10:39.816509 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.816490 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dqwnd\"" Jul 24 13:10:39.817538 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.817526 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tttb" Jul 24 13:10:39.821810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.821795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8tx" Jul 24 13:10:39.824848 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.824830 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:39.915277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.909889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.915277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.909920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:39.915277 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:39.910049 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 24 13:10:39.915277 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:10:39.910102 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls podName:495efb19-5d76-45a8-9f29-3dfca443e378 nodeName:}" failed. No retries permitted until 2026-07-24 13:10:40.91008359 +0000 UTC m=+67.509328781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-m6brm" (UID: "495efb19-5d76-45a8-9f29-3dfca443e378") : secret "kube-state-metrics-tls" not found Jul 24 13:10:39.915277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.912351 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/14a76ed7-c3d3-4843-9f7b-3f0619399d57-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-pzpv5\" (UID: \"14a76ed7-c3d3-4843-9f7b-3f0619399d57\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:39.960972 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:39.960945 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7tttb"] Jul 24 13:10:39.964608 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:39.964582 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c34749_5c39_4255_8891_20979a6cff44.slice/crio-94bf69ccd7dac7242d1b3eb0eaf8914461c0e199d28cd29540d4982734ab0e7b WatchSource:0}: Error finding container 94bf69ccd7dac7242d1b3eb0eaf8914461c0e199d28cd29540d4982734ab0e7b: Status 404 returned error can't find the container with id 94bf69ccd7dac7242d1b3eb0eaf8914461c0e199d28cd29540d4982734ab0e7b Jul 24 13:10:40.141190 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.141118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" Jul 24 13:10:40.193897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.193839 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8ncd6"] Jul 24 13:10:40.197859 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.197597 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dv8tx"] Jul 24 13:10:40.197859 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:40.197833 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb52081_e188_4f54_9448_cafd8edb1c96.slice/crio-9bf57b798a2a9386ae4d030969bb9523d84ca5e6f244e7f8d2125a99039f50ac WatchSource:0}: Error finding container 9bf57b798a2a9386ae4d030969bb9523d84ca5e6f244e7f8d2125a99039f50ac: Status 404 returned error can't find the container with id 9bf57b798a2a9386ae4d030969bb9523d84ca5e6f244e7f8d2125a99039f50ac Jul 24 13:10:40.200310 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:40.200119 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277edfa4_9e9a_4177_b38d_1bdffa4457ed.slice/crio-1545a5823e3106d337cdd4e7e1773c729736e9b019ef7891bbad274ff90ee3b4 WatchSource:0}: Error finding container 1545a5823e3106d337cdd4e7e1773c729736e9b019ef7891bbad274ff90ee3b4: Status 404 returned error can't find the container with id 1545a5823e3106d337cdd4e7e1773c729736e9b019ef7891bbad274ff90ee3b4 Jul 24 13:10:40.208590 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.208464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dv8tx" event={"ID":"277edfa4-9e9a-4177-b38d-1bdffa4457ed","Type":"ContainerStarted","Data":"1545a5823e3106d337cdd4e7e1773c729736e9b019ef7891bbad274ff90ee3b4"} Jul 24 13:10:40.210076 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.210052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ncd6" event={"ID":"7eb52081-e188-4f54-9448-cafd8edb1c96","Type":"ContainerStarted","Data":"9bf57b798a2a9386ae4d030969bb9523d84ca5e6f244e7f8d2125a99039f50ac"} Jul 24 13:10:40.211577 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.211539 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tttb" event={"ID":"79c34749-5c39-4255-8891-20979a6cff44","Type":"ContainerStarted","Data":"94bf69ccd7dac7242d1b3eb0eaf8914461c0e199d28cd29540d4982734ab0e7b"} Jul 24 13:10:40.212958 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.212939 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h55pl" event={"ID":"e09e824f-b26a-44b8-aff1-c7f2caf46b9a","Type":"ContainerStarted","Data":"c9de06cd8ea1b046700df090da53265f0bec5e64eaad0a9ce27dfc0348cd4de3"} Jul 24 13:10:40.277765 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.277731 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5"] Jul 24 13:10:40.281440 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:40.281405 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a76ed7_c3d3_4843_9f7b_3f0619399d57.slice/crio-ae02808b342350078b7b8740d05eba5f9e1bc702020478c14c33e4442f305099 WatchSource:0}: Error finding container ae02808b342350078b7b8740d05eba5f9e1bc702020478c14c33e4442f305099: Status 404 returned error can't find the container with id ae02808b342350078b7b8740d05eba5f9e1bc702020478c14c33e4442f305099 Jul 24 13:10:40.295763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.295742 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:10:40.299709 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.299619 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.302228 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.302207 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-2rfrk\"" Jul 24 13:10:40.302329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.302241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.302981 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.302998 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.303016 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.303430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.303531 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.303626 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 24 13:10:40.303778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.303778 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 24 13:10:40.304205 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.303996 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 24 13:10:40.316682 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.316656 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:10:40.413886 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413826 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.413886 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5d2r\" (UniqueName: \"kubernetes.io/projected/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-kube-api-access-q5d2r\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-web-config\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413936 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.413993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.414012 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-config-out\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414057 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.414042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.414072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.414114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.414152 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.414321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.414175 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515417 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515586 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515586 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515689 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515689 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515677 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5d2r\" (UniqueName: \"kubernetes.io/projected/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-kube-api-access-q5d2r\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-web-config\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515782 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515944 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515944 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515944 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-config-out\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.515944 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.516142 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.515958 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.516516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.516200 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.516516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.516302 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.517806 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.517762 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.518666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.518619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.518815 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.518773 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.519202 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.519177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-web-config\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.519386 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.519364 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.519497 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.519394 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.519583 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.519510 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-config-volume\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.519662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.519644 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-config-out\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.519939 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.519919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.520391 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.520375 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.523802 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.523784 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5d2r\" (UniqueName: \"kubernetes.io/projected/a4fa6fdf-b2e8-414b-826b-b7cb084a6194-kube-api-access-q5d2r\") pod \"alertmanager-main-0\" (UID: \"a4fa6fdf-b2e8-414b-826b-b7cb084a6194\") " pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.626253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.626232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 24 13:10:40.764019 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.763987 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 24 13:10:40.766655 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:40.766631 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fa6fdf_b2e8_414b_826b_b7cb084a6194.slice/crio-d069cc64cc8ddbde3827d14999d6074961e1b90a5044bc0357fd28f13077d294 WatchSource:0}: Error finding container d069cc64cc8ddbde3827d14999d6074961e1b90a5044bc0357fd28f13077d294: Status 404 returned error can't find the container with id d069cc64cc8ddbde3827d14999d6074961e1b90a5044bc0357fd28f13077d294 Jul 24 13:10:40.920475 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.920441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:40.923470 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:40.923421 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/495efb19-5d76-45a8-9f29-3dfca443e378-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-m6brm\" (UID: \"495efb19-5d76-45a8-9f29-3dfca443e378\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:41.062343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.062264 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" Jul 24 13:10:41.219921 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.219885 2562 generic.go:356] "Generic (PLEG): container finished" podID="e09e824f-b26a-44b8-aff1-c7f2caf46b9a" containerID="2db34034844da1b5a1715366e400b7fa1aa9243dc0be000012200140a5fb0ca3" exitCode=0 Jul 24 13:10:41.220149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.219986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h55pl" event={"ID":"e09e824f-b26a-44b8-aff1-c7f2caf46b9a","Type":"ContainerDied","Data":"2db34034844da1b5a1715366e400b7fa1aa9243dc0be000012200140a5fb0ca3"} Jul 24 13:10:41.224147 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.224043 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" event={"ID":"14a76ed7-c3d3-4843-9f7b-3f0619399d57","Type":"ContainerStarted","Data":"c2cc8296cd55aa190c9365572932db720a4ebf71da98c32ba00981a0fa64e129"} Jul 24 13:10:41.224147 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.224103 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" event={"ID":"14a76ed7-c3d3-4843-9f7b-3f0619399d57","Type":"ContainerStarted","Data":"fea381e2fb11df55cbd4b61cd80541bcc17f0d099bc5d2a2d8634fc45cb15280"} Jul 24 13:10:41.224147 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.224130 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" event={"ID":"14a76ed7-c3d3-4843-9f7b-3f0619399d57","Type":"ContainerStarted","Data":"ae02808b342350078b7b8740d05eba5f9e1bc702020478c14c33e4442f305099"} Jul 24 13:10:41.225817 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.225793 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"d069cc64cc8ddbde3827d14999d6074961e1b90a5044bc0357fd28f13077d294"} Jul 24 13:10:41.256336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:41.256306 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm"] Jul 24 13:10:41.627254 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:41.627153 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495efb19_5d76_45a8_9f29_3dfca443e378.slice/crio-fb46ef75f4fda8fe9d9fb2883be8775055c2a8b4ff8eaa99fe8cb7b326a47644 WatchSource:0}: Error finding container fb46ef75f4fda8fe9d9fb2883be8775055c2a8b4ff8eaa99fe8cb7b326a47644: Status 404 returned error can't find the container with id fb46ef75f4fda8fe9d9fb2883be8775055c2a8b4ff8eaa99fe8cb7b326a47644 Jul 24 13:10:42.233074 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:42.232817 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" event={"ID":"495efb19-5d76-45a8-9f29-3dfca443e378","Type":"ContainerStarted","Data":"fb46ef75f4fda8fe9d9fb2883be8775055c2a8b4ff8eaa99fe8cb7b326a47644"} Jul 24 13:10:43.239241 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.239197 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tttb" event={"ID":"79c34749-5c39-4255-8891-20979a6cff44","Type":"ContainerStarted","Data":"ec4e65654694acd425d15d67892b250d570605c87ab1cc0a7b0e76995a76fc58"} Jul 24 13:10:43.239736 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.239320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tttb" event={"ID":"79c34749-5c39-4255-8891-20979a6cff44","Type":"ContainerStarted","Data":"75a492a77805a16225abc46f4ee9939c8f9a2697c2c1085c8daed081a161eb06"} Jul 24 13:10:43.242579 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.242530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h55pl" event={"ID":"e09e824f-b26a-44b8-aff1-c7f2caf46b9a","Type":"ContainerStarted","Data":"965039a0878dafff3ce916be6fe407971d38b61a6bc4875cbdccecf0a42b6883"} Jul 24 13:10:43.242703 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.242602 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h55pl" event={"ID":"e09e824f-b26a-44b8-aff1-c7f2caf46b9a","Type":"ContainerStarted","Data":"3728e88d5de1d3ae3903f33ca5e17102a2a1d900f1bc32fea238dd27b3c0ad41"} Jul 24 13:10:43.245296 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.245268 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" event={"ID":"14a76ed7-c3d3-4843-9f7b-3f0619399d57","Type":"ContainerStarted","Data":"9a0716a15e4eb65f4d0f5eb7fc9c4aac243adbd8e7b962f748a7c46ef65f13a0"} Jul 24 13:10:43.247250 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.247176 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"31444416599b79312f9c71e72ca56c6f56b3a33f1f725cabbb544aabf99d3e3b"} Jul 24 13:10:43.255605 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.255535 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7tttb" podStartSLOduration=66.667093585 podStartE2EDuration="1m9.255518493s" totalImagesPullingTime="2.588424908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:43.254763185 +0000 UTC m=+69.854008397" watchObservedRunningTime="2026-07-24 13:10:43.255518493 +0000 UTC m=+69.854763705" Jul 24 13:10:43.298361 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.298303 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-pzpv5" podStartSLOduration=2.324532085 podStartE2EDuration="4.298281944s" totalImagesPullingTime="1.973749859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:43.271713316 +0000 UTC m=+69.870958530" watchObservedRunningTime="2026-07-24 13:10:43.298281944 +0000 UTC m=+69.897527167" Jul 24 13:10:43.316342 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:43.316292 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h55pl" podStartSLOduration=3.227279391 podStartE2EDuration="4.316275318s" totalImagesPullingTime="1.018082097s" totalInitContainerRuntime="70.91383ms" isStatefulPod=true podCreationTimestamp="2026-07-24 13:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:43.315555328 +0000 UTC m=+69.914800535" watchObservedRunningTime="2026-07-24 13:10:43.316275318 +0000 UTC m=+69.915520531" Jul 24 13:10:44.005893 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.005860 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d85677d58-lvkhm"] Jul 24 13:10:44.009044 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.009024 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.021639 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.021617 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d85677d58-lvkhm"] Jul 24 13:10:44.050537 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-config\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.050686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-oauth-config\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.050686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-trusted-ca-bundle\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.050686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050656 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-oauth-serving-cert\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.050686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-service-ca\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.050878 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-serving-cert\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.050878 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.050777 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv48d\" (UniqueName: \"kubernetes.io/projected/d693ddf8-4db7-45d0-bceb-d14c92c59c33-kube-api-access-zv48d\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.151751 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.151668 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-config\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.151751 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.151728 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-oauth-config\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.151971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.151757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-trusted-ca-bundle\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.151971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.151784 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-oauth-serving-cert\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.152076 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.152044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-service-ca\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.152509 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.152453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-serving-cert\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.152641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.152515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv48d\" (UniqueName: \"kubernetes.io/projected/d693ddf8-4db7-45d0-bceb-d14c92c59c33-kube-api-access-zv48d\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.152641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.152616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-config\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.152750 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.152730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-oauth-serving-cert\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.156988 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.153095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-trusted-ca-bundle\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.156988 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.153228 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-service-ca\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.156988 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.155627 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-serving-cert\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.156988 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.155666 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-oauth-config\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.161130 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.161098 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv48d\" (UniqueName: \"kubernetes.io/projected/d693ddf8-4db7-45d0-bceb-d14c92c59c33-kube-api-access-zv48d\") pod \"console-d85677d58-lvkhm\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:44.252416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.252381 2562 generic.go:356] "Generic (PLEG): container finished" podID="a4fa6fdf-b2e8-414b-826b-b7cb084a6194" containerID="31444416599b79312f9c71e72ca56c6f56b3a33f1f725cabbb544aabf99d3e3b" exitCode=0 Jul 24 13:10:44.252879 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.252511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerDied","Data":"31444416599b79312f9c71e72ca56c6f56b3a33f1f725cabbb544aabf99d3e3b"} Jul 24 13:10:44.319380 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:44.319351 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:45.779013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:45.778333 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d85677d58-lvkhm"] Jul 24 13:10:45.779013 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:45.778724 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd693ddf8_4db7_45d0_bceb_d14c92c59c33.slice/crio-056c628a97d5b0f48ec0b1268194fe13808b20083b927686761fb5ceda441a0e WatchSource:0}: Error finding container 056c628a97d5b0f48ec0b1268194fe13808b20083b927686761fb5ceda441a0e: Status 404 returned error can't find the container with id 056c628a97d5b0f48ec0b1268194fe13808b20083b927686761fb5ceda441a0e Jul 24 13:10:46.261949 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.261816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dv8tx" event={"ID":"277edfa4-9e9a-4177-b38d-1bdffa4457ed","Type":"ContainerStarted","Data":"075ea4ca6778e77c0f8692d080a76eb8ae6a5f9242431fe5911d9a63104b5852"} Jul 24 13:10:46.264085 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.264056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" event={"ID":"495efb19-5d76-45a8-9f29-3dfca443e378","Type":"ContainerStarted","Data":"1f3819f243b0976daccdbcda88d65dd5a294743eb59ffbb8d0d78c985e6b6ab6"} Jul 24 13:10:46.264208 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.264096 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" event={"ID":"495efb19-5d76-45a8-9f29-3dfca443e378","Type":"ContainerStarted","Data":"facd8d4e5e010d0ab4a869b043402c44a43f5c9daab330ac3c1211ae80b8ac58"} Jul 24 13:10:46.264208 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.264117 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" event={"ID":"495efb19-5d76-45a8-9f29-3dfca443e378","Type":"ContainerStarted","Data":"d8d8894795bc4b26df8aab86d1e995c8b3c9326a0193f0693c0189f1eccace10"} Jul 24 13:10:46.265595 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.265572 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d85677d58-lvkhm" event={"ID":"d693ddf8-4db7-45d0-bceb-d14c92c59c33","Type":"ContainerStarted","Data":"ffdfb230246ce8d84fc6da4e4ee2ef2bdda4ec96de3d713bbd2e70c008149623"} Jul 24 13:10:46.265704 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.265611 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d85677d58-lvkhm" event={"ID":"d693ddf8-4db7-45d0-bceb-d14c92c59c33","Type":"ContainerStarted","Data":"056c628a97d5b0f48ec0b1268194fe13808b20083b927686761fb5ceda441a0e"} Jul 24 13:10:46.267474 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.267456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8ncd6" event={"ID":"7eb52081-e188-4f54-9448-cafd8edb1c96","Type":"ContainerStarted","Data":"e1f1f9886323cf0d677e437cd6faa4dc36ccb7206d4a7823a32b338665c21acd"} Jul 24 13:10:46.267610 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.267595 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:10:46.281178 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.281133 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dv8tx" podStartSLOduration=66.872348584 podStartE2EDuration="1m12.281115831s" totalImagesPullingTime="5.408767247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:46.280260049 +0000 UTC m=+72.879505263" watchObservedRunningTime="2026-07-24 13:10:46.281115831 +0000 UTC m=+72.880361043" Jul 24 13:10:46.297419 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.297378 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d85677d58-lvkhm" podStartSLOduration=3.297364484 podStartE2EDuration="3.297364484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:46.296861473 +0000 UTC m=+72.896106686" watchObservedRunningTime="2026-07-24 13:10:46.297364484 +0000 UTC m=+72.896609749" Jul 24 13:10:46.326832 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.326783 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-m6brm" podStartSLOduration=3.354602013 podStartE2EDuration="7.326763474s" totalImagesPullingTime="3.972161461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:46.319846208 +0000 UTC m=+72.919091420" watchObservedRunningTime="2026-07-24 13:10:46.326763474 +0000 UTC m=+72.926008687" Jul 24 13:10:46.342344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:46.342297 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8ncd6" podStartSLOduration=66.940877951 podStartE2EDuration="1m12.342281357s" totalImagesPullingTime="5.401403406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:46.341255412 +0000 UTC m=+72.940500622" watchObservedRunningTime="2026-07-24 13:10:46.342281357 +0000 UTC m=+72.941526570" Jul 24 13:10:47.274776 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:47.274741 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"a80189eeffdc34eb9d25d598e8a1c74f789da7d9c72e585068f5a3ffb84ebdff"} Jul 24 13:10:47.275234 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:47.274830 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"a00b2a67e3b6b2dc34ba21e0f17decdcf84a483f0c2f79cd2b4213c149f94be5"} Jul 24 13:10:47.275234 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:47.274862 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"ab89ee84de7f09430f3d91d6e8e62816f3339ed534d6556416a59534614eca10"} Jul 24 13:10:47.275234 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:47.274887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"b12235bbb4bef898ae12daf6213622d0042cb3ca859e12c64481acb11df472dc"} Jul 24 13:10:47.275234 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:47.274917 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"5ab40dcc0721a64be9955f2e23e1d0cf81f1e1a3e679081baca9c8cab1398792"} Jul 24 13:10:48.282021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:48.281982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a4fa6fdf-b2e8-414b-826b-b7cb084a6194","Type":"ContainerStarted","Data":"dfcd37d167a146a84fafdf6a449ebaf860f40ce15ea28dd78c442bd1229176ab"} Jul 24 13:10:48.318705 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:48.318539 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.328154724 podStartE2EDuration="8.318518989s" totalImagesPullingTime="4.965595692s" totalInitContainerRuntime="24.768573ms" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:48.318041256 +0000 UTC m=+74.917286468" watchObservedRunningTime="2026-07-24 13:10:48.318518989 +0000 UTC m=+74.917764208" Jul 24 13:10:50.788421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:50.788380 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d85677d58-lvkhm"] Jul 24 13:10:50.838201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:50.838165 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f48d6d4b5-wwlsn"] Jul 24 13:10:50.975960 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:50.975924 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f48d6d4b5-wwlsn"] Jul 24 13:10:50.976101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:50.976040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112690 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-trusted-ca-bundle\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112690 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-oauth-serving-cert\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112690 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-config\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112957 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112753 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-serving-cert\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112957 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-oauth-config\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112957 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbwb9\" (UniqueName: \"kubernetes.io/projected/9f6f88cb-1813-404f-8cdd-2e6aa4276658-kube-api-access-sbwb9\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.112957 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.112854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-service-ca\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213493 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-trusted-ca-bundle\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213660 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213502 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-oauth-serving-cert\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213660 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-config\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213660 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-serving-cert\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213787 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-oauth-config\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213787 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbwb9\" (UniqueName: \"kubernetes.io/projected/9f6f88cb-1813-404f-8cdd-2e6aa4276658-kube-api-access-sbwb9\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.213787 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.213763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-service-ca\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.214324 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.214291 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-config\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.214447 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.214428 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-service-ca\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.214515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.214472 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-trusted-ca-bundle\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.214639 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.214615 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-oauth-serving-cert\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.216018 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.215997 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-oauth-config\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.216269 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.216247 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-serving-cert\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.222616 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.222595 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbwb9\" (UniqueName: \"kubernetes.io/projected/9f6f88cb-1813-404f-8cdd-2e6aa4276658-kube-api-access-sbwb9\") pod \"console-6f48d6d4b5-wwlsn\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.284712 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.284687 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:10:51.431214 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:51.431189 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f48d6d4b5-wwlsn"] Jul 24 13:10:51.433390 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:10:51.433366 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6f88cb_1813_404f_8cdd_2e6aa4276658.slice/crio-b9f4f96267c4ec14ac1225629ee089171ca19ae30f4f252135ce2c4cc37b814f WatchSource:0}: Error finding container b9f4f96267c4ec14ac1225629ee089171ca19ae30f4f252135ce2c4cc37b814f: Status 404 returned error can't find the container with id b9f4f96267c4ec14ac1225629ee089171ca19ae30f4f252135ce2c4cc37b814f Jul 24 13:10:52.298393 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:52.298355 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f48d6d4b5-wwlsn" event={"ID":"9f6f88cb-1813-404f-8cdd-2e6aa4276658","Type":"ContainerStarted","Data":"72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee"} Jul 24 13:10:52.298754 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:52.298412 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f48d6d4b5-wwlsn" event={"ID":"9f6f88cb-1813-404f-8cdd-2e6aa4276658","Type":"ContainerStarted","Data":"b9f4f96267c4ec14ac1225629ee089171ca19ae30f4f252135ce2c4cc37b814f"} Jul 24 13:10:52.317124 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:52.317081 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f48d6d4b5-wwlsn" podStartSLOduration=2.317064168 podStartE2EDuration="2.317064168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:10:52.314666542 +0000 UTC m=+78.913911753" watchObservedRunningTime="2026-07-24 13:10:52.317064168 +0000 UTC m=+78.916309379" Jul 24 13:10:54.320128 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:54.320097 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:10:59.714763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:10:59.714652 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xk4j5_2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2/serve-healthcheck-canary/0.log" Jul 24 13:11:01.285590 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:01.285537 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:11:01.285954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:01.285612 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:11:01.290103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:01.290083 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:11:01.333943 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:01.333919 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:11:01.378483 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:01.378446 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df5db56cb-kk4sx"] Jul 24 13:11:15.807186 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:15.807144 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d85677d58-lvkhm" podUID="d693ddf8-4db7-45d0-bceb-d14c92c59c33" containerName="console" containerID="cri-o://ffdfb230246ce8d84fc6da4e4ee2ef2bdda4ec96de3d713bbd2e70c008149623" gracePeriod=15 Jul 24 13:11:16.390791 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.390755 2562 generic.go:356] "Generic (PLEG): container finished" podID="d693ddf8-4db7-45d0-bceb-d14c92c59c33" containerID="ffdfb230246ce8d84fc6da4e4ee2ef2bdda4ec96de3d713bbd2e70c008149623" exitCode=0 Jul 24 13:11:16.390947 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.390800 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d85677d58-lvkhm" event={"ID":"d693ddf8-4db7-45d0-bceb-d14c92c59c33","Type":"ContainerDied","Data":"ffdfb230246ce8d84fc6da4e4ee2ef2bdda4ec96de3d713bbd2e70c008149623"} Jul 24 13:11:16.568862 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.568837 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:11:16.720029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720006 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-oauth-serving-cert\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720070 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-serving-cert\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720095 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-config\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720128 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv48d\" (UniqueName: \"kubernetes.io/projected/d693ddf8-4db7-45d0-bceb-d14c92c59c33-kube-api-access-zv48d\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-service-ca\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720321 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-trusted-ca-bundle\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720401 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720370 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-oauth-config\") pod \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\" (UID: \"d693ddf8-4db7-45d0-bceb-d14c92c59c33\") " Jul 24 13:11:16.720510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720448 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:16.720510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720502 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-service-ca" (OuterVolumeSpecName: "service-ca") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:16.720772 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720669 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-config" (OuterVolumeSpecName: "console-config") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:16.720772 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720746 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-oauth-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:16.720772 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720765 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:16.720890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720779 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:16.720890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.720782 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:16.722458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.722436 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:11:16.722536 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.722501 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d693ddf8-4db7-45d0-bceb-d14c92c59c33-kube-api-access-zv48d" (OuterVolumeSpecName: "kube-api-access-zv48d") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "kube-api-access-zv48d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:11:16.722536 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.722513 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d693ddf8-4db7-45d0-bceb-d14c92c59c33" (UID: "d693ddf8-4db7-45d0-bceb-d14c92c59c33"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:11:16.821658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.821637 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:16.821658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.821657 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv48d\" (UniqueName: \"kubernetes.io/projected/d693ddf8-4db7-45d0-bceb-d14c92c59c33-kube-api-access-zv48d\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:16.821984 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.821667 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d693ddf8-4db7-45d0-bceb-d14c92c59c33-trusted-ca-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:16.821984 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:16.821676 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d693ddf8-4db7-45d0-bceb-d14c92c59c33-console-oauth-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:17.277223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.277198 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8ncd6" Jul 24 13:11:17.395654 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.395625 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d85677d58-lvkhm" Jul 24 13:11:17.395654 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.395629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d85677d58-lvkhm" event={"ID":"d693ddf8-4db7-45d0-bceb-d14c92c59c33","Type":"ContainerDied","Data":"056c628a97d5b0f48ec0b1268194fe13808b20083b927686761fb5ceda441a0e"} Jul 24 13:11:17.395878 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.395680 2562 scope.go:117] "RemoveContainer" containerID="ffdfb230246ce8d84fc6da4e4ee2ef2bdda4ec96de3d713bbd2e70c008149623" Jul 24 13:11:17.415167 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.415146 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d85677d58-lvkhm"] Jul 24 13:11:17.420237 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.420176 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d85677d58-lvkhm"] Jul 24 13:11:17.894899 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:17.894867 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d693ddf8-4db7-45d0-bceb-d14c92c59c33" path="/var/lib/kubelet/pods/d693ddf8-4db7-45d0-bceb-d14c92c59c33/volumes" Jul 24 13:11:26.397197 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.397156 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-df5db56cb-kk4sx" podUID="0ebfd84e-f4dc-4314-b9e7-4356005a892e" containerName="console" containerID="cri-o://31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883" gracePeriod=15 Jul 24 13:11:26.630348 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.630327 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:11:26.694946 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.694920 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzcsl\" (UniqueName: \"kubernetes.io/projected/0ebfd84e-f4dc-4314-b9e7-4356005a892e-kube-api-access-gzcsl\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.694964 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-service-ca\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.694990 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-oauth-config\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695061 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-oauth-serving-cert\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695243 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695087 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-serving-cert\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695243 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695110 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-trusted-ca-bundle\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695243 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695145 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-config\") pod \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\" (UID: \"0ebfd84e-f4dc-4314-b9e7-4356005a892e\") " Jul 24 13:11:26.695520 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695391 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-service-ca" (OuterVolumeSpecName: "service-ca") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:26.695658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695527 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:26.695725 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695669 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-config" (OuterVolumeSpecName: "console-config") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:26.695725 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.695680 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:11:26.697085 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.697065 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ebfd84e-f4dc-4314-b9e7-4356005a892e-kube-api-access-gzcsl" (OuterVolumeSpecName: "kube-api-access-gzcsl") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "kube-api-access-gzcsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:11:26.697232 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.697209 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:11:26.697315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.697290 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0ebfd84e-f4dc-4314-b9e7-4356005a892e" (UID: "0ebfd84e-f4dc-4314-b9e7-4356005a892e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:11:26.795701 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795655 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzcsl\" (UniqueName: \"kubernetes.io/projected/0ebfd84e-f4dc-4314-b9e7-4356005a892e-kube-api-access-gzcsl\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:26.795701 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795694 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:26.795701 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795704 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-oauth-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:26.795701 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795712 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-oauth-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:26.795954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795721 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:26.795954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795729 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-trusted-ca-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:26.795954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:26.795737 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ebfd84e-f4dc-4314-b9e7-4356005a892e-console-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:11:27.433641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.433608 2562 generic.go:356] "Generic (PLEG): container finished" podID="0ebfd84e-f4dc-4314-b9e7-4356005a892e" containerID="31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883" exitCode=0 Jul 24 13:11:27.434033 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.433656 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df5db56cb-kk4sx" Jul 24 13:11:27.434033 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.433668 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df5db56cb-kk4sx" event={"ID":"0ebfd84e-f4dc-4314-b9e7-4356005a892e","Type":"ContainerDied","Data":"31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883"} Jul 24 13:11:27.434033 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.433707 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df5db56cb-kk4sx" event={"ID":"0ebfd84e-f4dc-4314-b9e7-4356005a892e","Type":"ContainerDied","Data":"a5339fedec9cb47c2d57d32263ad149f8e56f027ca18537a54fac0b4e3fd4319"} Jul 24 13:11:27.434033 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.433727 2562 scope.go:117] "RemoveContainer" containerID="31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883" Jul 24 13:11:27.441739 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.441723 2562 scope.go:117] "RemoveContainer" containerID="31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883" Jul 24 13:11:27.441971 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:11:27.441949 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883\": container with ID starting with 31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883 not found: ID does not exist" containerID="31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883" Jul 24 13:11:27.442014 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.441979 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883"} err="failed to get container status \"31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883\": rpc error: code = NotFound desc = could not find container \"31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883\": container with ID starting with 31de64fe50c7a7596b9bd0976e0343150388e2415662369e44b818684bb37883 not found: ID does not exist" Jul 24 13:11:27.452888 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.452869 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df5db56cb-kk4sx"] Jul 24 13:11:27.456724 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.456703 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-df5db56cb-kk4sx"] Jul 24 13:11:27.894841 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:11:27.894762 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ebfd84e-f4dc-4314-b9e7-4356005a892e" path="/var/lib/kubelet/pods/0ebfd84e-f4dc-4314-b9e7-4356005a892e/volumes" Jul 24 13:12:11.032845 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.032812 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-564899d58c-jssfd"] Jul 24 13:12:11.033420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.033176 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d693ddf8-4db7-45d0-bceb-d14c92c59c33" containerName="console" Jul 24 13:12:11.033420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.033192 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d693ddf8-4db7-45d0-bceb-d14c92c59c33" containerName="console" Jul 24 13:12:11.033420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.033219 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ebfd84e-f4dc-4314-b9e7-4356005a892e" containerName="console" Jul 24 13:12:11.033420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.033228 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ebfd84e-f4dc-4314-b9e7-4356005a892e" containerName="console" Jul 24 13:12:11.033420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.033296 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ebfd84e-f4dc-4314-b9e7-4356005a892e" containerName="console" Jul 24 13:12:11.033420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.033310 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d693ddf8-4db7-45d0-bceb-d14c92c59c33" containerName="console" Jul 24 13:12:11.035188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.035168 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.045948 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.045927 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-564899d58c-jssfd"] Jul 24 13:12:11.081389 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-oauth-config\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.081510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-trusted-ca-bundle\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.081510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-serving-cert\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.081510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp5gd\" (UniqueName: \"kubernetes.io/projected/221f5a62-7982-4798-b116-29dc599e5dca-kube-api-access-fp5gd\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.081510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081476 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-oauth-serving-cert\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.081510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-service-ca\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.081698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.081519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-console-config\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182826 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-oauth-config\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182832 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-trusted-ca-bundle\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-serving-cert\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp5gd\" (UniqueName: \"kubernetes.io/projected/221f5a62-7982-4798-b116-29dc599e5dca-kube-api-access-fp5gd\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-oauth-serving-cert\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182962 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-service-ca\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.182995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.182989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-console-config\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.183701 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.183678 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-oauth-serving-cert\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.183813 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.183797 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-console-config\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.183895 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.183871 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-service-ca\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.183951 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.183921 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-trusted-ca-bundle\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.185134 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.185115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-oauth-config\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.185288 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.185272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-serving-cert\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.190969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.190948 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp5gd\" (UniqueName: \"kubernetes.io/projected/221f5a62-7982-4798-b116-29dc599e5dca-kube-api-access-fp5gd\") pod \"console-564899d58c-jssfd\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.344272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.344198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:11.466369 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.466336 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-564899d58c-jssfd"] Jul 24 13:12:11.469660 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:12:11.469635 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221f5a62_7982_4798_b116_29dc599e5dca.slice/crio-074bf18047eacf37f2b6490304e60744bcc2d9d028f6c668bb63ad04910a9cbd WatchSource:0}: Error finding container 074bf18047eacf37f2b6490304e60744bcc2d9d028f6c668bb63ad04910a9cbd: Status 404 returned error can't find the container with id 074bf18047eacf37f2b6490304e60744bcc2d9d028f6c668bb63ad04910a9cbd Jul 24 13:12:11.591122 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.591085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564899d58c-jssfd" event={"ID":"221f5a62-7982-4798-b116-29dc599e5dca","Type":"ContainerStarted","Data":"1c914d6251fa2863cf8b02611d9b8f67568f7b25952555d2924213978da81380"} Jul 24 13:12:11.591274 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.591142 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564899d58c-jssfd" event={"ID":"221f5a62-7982-4798-b116-29dc599e5dca","Type":"ContainerStarted","Data":"074bf18047eacf37f2b6490304e60744bcc2d9d028f6c668bb63ad04910a9cbd"} Jul 24 13:12:11.610124 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:11.610035 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-564899d58c-jssfd" podStartSLOduration=0.610018413 podStartE2EDuration="610.018413ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:12:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:12:11.609283836 +0000 UTC m=+158.208529060" watchObservedRunningTime="2026-07-24 13:12:11.610018413 +0000 UTC m=+158.209263623" Jul 24 13:12:21.345291 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:21.345240 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:21.345729 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:21.345326 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:21.349662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:21.349638 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:21.629495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:21.629425 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:12:21.679217 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:21.679182 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f48d6d4b5-wwlsn"] Jul 24 13:12:46.699166 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:46.699112 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f48d6d4b5-wwlsn" podUID="9f6f88cb-1813-404f-8cdd-2e6aa4276658" containerName="console" containerID="cri-o://72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee" gracePeriod=15 Jul 24 13:12:46.931007 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:46.930986 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:12:47.042219 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042142 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-oauth-config\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042219 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042174 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-service-ca\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042219 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-config\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042228 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-trusted-ca-bundle\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042261 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbwb9\" (UniqueName: \"kubernetes.io/projected/9f6f88cb-1813-404f-8cdd-2e6aa4276658-kube-api-access-sbwb9\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042281 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-serving-cert\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042314 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-oauth-serving-cert\") pod \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\" (UID: \"9f6f88cb-1813-404f-8cdd-2e6aa4276658\") " Jul 24 13:12:47.042724 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042697 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-config" (OuterVolumeSpecName: "console-config") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:12:47.042781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042741 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-service-ca" (OuterVolumeSpecName: "service-ca") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:12:47.042824 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042782 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:12:47.042824 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.042789 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:12:47.044530 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.044503 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:12:47.044640 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.044596 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6f88cb-1813-404f-8cdd-2e6aa4276658-kube-api-access-sbwb9" (OuterVolumeSpecName: "kube-api-access-sbwb9") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "kube-api-access-sbwb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:12:47.044640 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.044610 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9f6f88cb-1813-404f-8cdd-2e6aa4276658" (UID: "9f6f88cb-1813-404f-8cdd-2e6aa4276658"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:12:47.143677 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143651 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-trusted-ca-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.143677 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143673 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbwb9\" (UniqueName: \"kubernetes.io/projected/9f6f88cb-1813-404f-8cdd-2e6aa4276658-kube-api-access-sbwb9\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.143810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143684 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.143810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143692 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-oauth-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.143810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143702 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-oauth-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.143810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143709 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.143810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.143718 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9f6f88cb-1813-404f-8cdd-2e6aa4276658-console-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:12:47.713124 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.713088 2562 generic.go:356] "Generic (PLEG): container finished" podID="9f6f88cb-1813-404f-8cdd-2e6aa4276658" containerID="72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee" exitCode=0 Jul 24 13:12:47.713469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.713135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f48d6d4b5-wwlsn" event={"ID":"9f6f88cb-1813-404f-8cdd-2e6aa4276658","Type":"ContainerDied","Data":"72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee"} Jul 24 13:12:47.713469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.713155 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f48d6d4b5-wwlsn" Jul 24 13:12:47.713469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.713214 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f48d6d4b5-wwlsn" event={"ID":"9f6f88cb-1813-404f-8cdd-2e6aa4276658","Type":"ContainerDied","Data":"b9f4f96267c4ec14ac1225629ee089171ca19ae30f4f252135ce2c4cc37b814f"} Jul 24 13:12:47.713469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.713243 2562 scope.go:117] "RemoveContainer" containerID="72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee" Jul 24 13:12:47.721343 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.721327 2562 scope.go:117] "RemoveContainer" containerID="72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee" Jul 24 13:12:47.721612 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:12:47.721585 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee\": container with ID starting with 72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee not found: ID does not exist" containerID="72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee" Jul 24 13:12:47.721675 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.721624 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee"} err="failed to get container status \"72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee\": rpc error: code = NotFound desc = could not find container \"72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee\": container with ID starting with 72f8d1aafbf7f0e35664e38420c0a81ef16b0478ee6edf5be86eaa66c479b9ee not found: ID does not exist" Jul 24 13:12:47.732660 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.732640 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f48d6d4b5-wwlsn"] Jul 24 13:12:47.737060 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.737042 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f48d6d4b5-wwlsn"] Jul 24 13:12:47.895076 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:47.895043 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6f88cb-1813-404f-8cdd-2e6aa4276658" path="/var/lib/kubelet/pods/9f6f88cb-1813-404f-8cdd-2e6aa4276658/volumes" Jul 24 13:12:50.957062 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.957033 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b"] Jul 24 13:12:50.957694 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.957440 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6f88cb-1813-404f-8cdd-2e6aa4276658" containerName="console" Jul 24 13:12:50.957694 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.957458 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6f88cb-1813-404f-8cdd-2e6aa4276658" containerName="console" Jul 24 13:12:50.957694 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.957540 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6f88cb-1813-404f-8cdd-2e6aa4276658" containerName="console" Jul 24 13:12:50.960144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.960125 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:50.962337 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.962315 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 13:12:50.962427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.962392 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wxfsb\"" Jul 24 13:12:50.962472 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.962449 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 13:12:50.969139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:50.969120 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b"] Jul 24 13:12:51.072246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.072213 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.072246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.072249 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfhkw\" (UniqueName: \"kubernetes.io/projected/2e370548-80b9-448a-bde8-d025ade45fb7-kube-api-access-lfhkw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.072496 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.072278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.172876 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.172851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.173031 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.172891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfhkw\" (UniqueName: \"kubernetes.io/projected/2e370548-80b9-448a-bde8-d025ade45fb7-kube-api-access-lfhkw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.173031 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.172931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.173314 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.173275 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.173314 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.173285 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.181917 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.181895 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfhkw\" (UniqueName: \"kubernetes.io/projected/2e370548-80b9-448a-bde8-d025ade45fb7-kube-api-access-lfhkw\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.269368 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.269278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:12:51.392320 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.392290 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b"] Jul 24 13:12:51.395022 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:12:51.394988 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e370548_80b9_448a_bde8_d025ade45fb7.slice/crio-5de2b37cdf6769611432211c8837f73f9e2a6ec45f3a15b3b793a9e8d2d2f563 WatchSource:0}: Error finding container 5de2b37cdf6769611432211c8837f73f9e2a6ec45f3a15b3b793a9e8d2d2f563: Status 404 returned error can't find the container with id 5de2b37cdf6769611432211c8837f73f9e2a6ec45f3a15b3b793a9e8d2d2f563 Jul 24 13:12:51.728334 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:51.728302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" event={"ID":"2e370548-80b9-448a-bde8-d025ade45fb7","Type":"ContainerStarted","Data":"5de2b37cdf6769611432211c8837f73f9e2a6ec45f3a15b3b793a9e8d2d2f563"} Jul 24 13:12:58.758748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:58.758712 2562 generic.go:356] "Generic (PLEG): container finished" podID="2e370548-80b9-448a-bde8-d025ade45fb7" containerID="a2796b69a42a597905e6414e217df2052bd07c7d82f756f4b02a7e468f950134" exitCode=0 Jul 24 13:12:58.758748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:12:58.758751 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" event={"ID":"2e370548-80b9-448a-bde8-d025ade45fb7","Type":"ContainerDied","Data":"a2796b69a42a597905e6414e217df2052bd07c7d82f756f4b02a7e468f950134"} Jul 24 13:13:01.770594 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:01.770537 2562 generic.go:356] "Generic (PLEG): container finished" podID="2e370548-80b9-448a-bde8-d025ade45fb7" containerID="6649b29d2eae9f4e90d4b742b89a2b1cf0c72823697482a69c06cd7f2c2db53a" exitCode=0 Jul 24 13:13:01.770972 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:01.770620 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" event={"ID":"2e370548-80b9-448a-bde8-d025ade45fb7","Type":"ContainerDied","Data":"6649b29d2eae9f4e90d4b742b89a2b1cf0c72823697482a69c06cd7f2c2db53a"} Jul 24 13:13:09.803784 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:09.803740 2562 generic.go:356] "Generic (PLEG): container finished" podID="2e370548-80b9-448a-bde8-d025ade45fb7" containerID="8b5c6663df1eda2741cfbc585f27fe06e685042c5a3ac78e1878b59baf0d9647" exitCode=0 Jul 24 13:13:09.804126 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:09.803807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" event={"ID":"2e370548-80b9-448a-bde8-d025ade45fb7","Type":"ContainerDied","Data":"8b5c6663df1eda2741cfbc585f27fe06e685042c5a3ac78e1878b59baf0d9647"} Jul 24 13:13:10.927778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:10.927754 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:13:11.033203 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.033175 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-util\") pod \"2e370548-80b9-448a-bde8-d025ade45fb7\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " Jul 24 13:13:11.033329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.033228 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfhkw\" (UniqueName: \"kubernetes.io/projected/2e370548-80b9-448a-bde8-d025ade45fb7-kube-api-access-lfhkw\") pod \"2e370548-80b9-448a-bde8-d025ade45fb7\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " Jul 24 13:13:11.033329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.033275 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-bundle\") pod \"2e370548-80b9-448a-bde8-d025ade45fb7\" (UID: \"2e370548-80b9-448a-bde8-d025ade45fb7\") " Jul 24 13:13:11.034079 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.034052 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-bundle" (OuterVolumeSpecName: "bundle") pod "2e370548-80b9-448a-bde8-d025ade45fb7" (UID: "2e370548-80b9-448a-bde8-d025ade45fb7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:13:11.035423 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.035405 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e370548-80b9-448a-bde8-d025ade45fb7-kube-api-access-lfhkw" (OuterVolumeSpecName: "kube-api-access-lfhkw") pod "2e370548-80b9-448a-bde8-d025ade45fb7" (UID: "2e370548-80b9-448a-bde8-d025ade45fb7"). InnerVolumeSpecName "kube-api-access-lfhkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:13:11.038036 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.038017 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-util" (OuterVolumeSpecName: "util") pod "2e370548-80b9-448a-bde8-d025ade45fb7" (UID: "2e370548-80b9-448a-bde8-d025ade45fb7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:13:11.134413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.134345 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:11.134413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.134369 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfhkw\" (UniqueName: \"kubernetes.io/projected/2e370548-80b9-448a-bde8-d025ade45fb7-kube-api-access-lfhkw\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:11.134413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.134379 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e370548-80b9-448a-bde8-d025ade45fb7-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:11.812721 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.812686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" event={"ID":"2e370548-80b9-448a-bde8-d025ade45fb7","Type":"ContainerDied","Data":"5de2b37cdf6769611432211c8837f73f9e2a6ec45f3a15b3b793a9e8d2d2f563"} Jul 24 13:13:11.812721 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.812716 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e6754025n2b" Jul 24 13:13:11.812907 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:11.812737 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5de2b37cdf6769611432211c8837f73f9e2a6ec45f3a15b3b793a9e8d2d2f563" Jul 24 13:13:19.289645 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289615 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c"] Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289867 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="pull" Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289880 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="pull" Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289894 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="util" Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289900 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="util" Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289913 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="extract" Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289918 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="extract" Jul 24 13:13:19.290063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.289956 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e370548-80b9-448a-bde8-d025ade45fb7" containerName="extract" Jul 24 13:13:19.293314 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.293295 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.295615 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.295593 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:13:19.295837 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.295823 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 24 13:13:19.295903 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.295880 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-sbjjx\"" Jul 24 13:13:19.305347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.305324 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c"] Jul 24 13:13:19.399213 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.399189 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtj9p\" (UniqueName: \"kubernetes.io/projected/32232ede-a5de-42fd-b19c-d6e04f907292-kube-api-access-wtj9p\") pod \"cert-manager-operator-controller-manager-55c444bf86-f7c7c\" (UID: \"32232ede-a5de-42fd-b19c-d6e04f907292\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.399322 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.399222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32232ede-a5de-42fd-b19c-d6e04f907292-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-f7c7c\" (UID: \"32232ede-a5de-42fd-b19c-d6e04f907292\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.500502 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.500470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtj9p\" (UniqueName: \"kubernetes.io/projected/32232ede-a5de-42fd-b19c-d6e04f907292-kube-api-access-wtj9p\") pod \"cert-manager-operator-controller-manager-55c444bf86-f7c7c\" (UID: \"32232ede-a5de-42fd-b19c-d6e04f907292\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.500629 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.500516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32232ede-a5de-42fd-b19c-d6e04f907292-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-f7c7c\" (UID: \"32232ede-a5de-42fd-b19c-d6e04f907292\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.500887 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.500870 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32232ede-a5de-42fd-b19c-d6e04f907292-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-f7c7c\" (UID: \"32232ede-a5de-42fd-b19c-d6e04f907292\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.510305 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.510282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtj9p\" (UniqueName: \"kubernetes.io/projected/32232ede-a5de-42fd-b19c-d6e04f907292-kube-api-access-wtj9p\") pod \"cert-manager-operator-controller-manager-55c444bf86-f7c7c\" (UID: \"32232ede-a5de-42fd-b19c-d6e04f907292\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.602289 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.602231 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" Jul 24 13:13:19.725676 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.725649 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c"] Jul 24 13:13:19.729147 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:13:19.729113 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32232ede_a5de_42fd_b19c_d6e04f907292.slice/crio-c03add6c825efdb98755bbe560bca9e4690fd4cf4400cbd1e52899d01d7bfe24 WatchSource:0}: Error finding container c03add6c825efdb98755bbe560bca9e4690fd4cf4400cbd1e52899d01d7bfe24: Status 404 returned error can't find the container with id c03add6c825efdb98755bbe560bca9e4690fd4cf4400cbd1e52899d01d7bfe24 Jul 24 13:13:19.841570 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:19.841528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" event={"ID":"32232ede-a5de-42fd-b19c-d6e04f907292","Type":"ContainerStarted","Data":"c03add6c825efdb98755bbe560bca9e4690fd4cf4400cbd1e52899d01d7bfe24"} Jul 24 13:13:21.852088 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:21.852037 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" event={"ID":"32232ede-a5de-42fd-b19c-d6e04f907292","Type":"ContainerStarted","Data":"f66f6214c2abd77137c1afc83a5c56168999979b80b15cdb1a0b1783cf1c228a"} Jul 24 13:13:21.872146 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:21.872088 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-f7c7c" podStartSLOduration=1.221042541 podStartE2EDuration="2.872074121s" totalImagesPullingTime="1.65103158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:13:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:13:21.870982837 +0000 UTC m=+228.470228049" watchObservedRunningTime="2026-07-24 13:13:21.872074121 +0000 UTC m=+228.471319332" Jul 24 13:13:22.844417 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.844386 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44"] Jul 24 13:13:22.846451 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.846436 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:22.848868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.848845 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 13:13:22.848868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.848848 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wxfsb\"" Jul 24 13:13:22.849036 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.848848 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 13:13:22.856599 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.856580 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44"] Jul 24 13:13:22.928716 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.928690 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:22.928835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.928726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:22.928835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:22.928793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x6rd\" (UniqueName: \"kubernetes.io/projected/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-kube-api-access-7x6rd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.029186 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.029150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.029349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.029200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.029349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.029233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x6rd\" (UniqueName: \"kubernetes.io/projected/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-kube-api-access-7x6rd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.029621 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.029592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.029664 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.029619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.037186 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.037158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x6rd\" (UniqueName: \"kubernetes.io/projected/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-kube-api-access-7x6rd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.155416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.155344 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:23.283452 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.283425 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44"] Jul 24 13:13:23.286502 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:13:23.286473 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cd27cd_04fa_4db6_bcce_ad1c37cc88a9.slice/crio-6950d55b28d94bbf69aadcc4bb69128a163f22d15fbe69be10e4a652ea2b68f3 WatchSource:0}: Error finding container 6950d55b28d94bbf69aadcc4bb69128a163f22d15fbe69be10e4a652ea2b68f3: Status 404 returned error can't find the container with id 6950d55b28d94bbf69aadcc4bb69128a163f22d15fbe69be10e4a652ea2b68f3 Jul 24 13:13:23.860470 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.860433 2562 generic.go:356] "Generic (PLEG): container finished" podID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerID="b42f666b15e0bbf3636b096040cc3a4954a5213d71f1932e4e5cbd34a8488cda" exitCode=0 Jul 24 13:13:23.860901 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.860509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" event={"ID":"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9","Type":"ContainerDied","Data":"b42f666b15e0bbf3636b096040cc3a4954a5213d71f1932e4e5cbd34a8488cda"} Jul 24 13:13:23.860901 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:23.860567 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" event={"ID":"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9","Type":"ContainerStarted","Data":"6950d55b28d94bbf69aadcc4bb69128a163f22d15fbe69be10e4a652ea2b68f3"} Jul 24 13:13:24.142253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.142172 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fsztt"] Jul 24 13:13:24.144316 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.144300 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.146572 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.146534 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 13:13:24.146658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.146577 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 13:13:24.147566 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.147532 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vbnp6\"" Jul 24 13:13:24.154261 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.154242 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fsztt"] Jul 24 13:13:24.235639 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.235611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcafe656-9fdf-4609-9b2b-eae73b8e4677-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fsztt\" (UID: \"fcafe656-9fdf-4609-9b2b-eae73b8e4677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.235639 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.235640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lnj\" (UniqueName: \"kubernetes.io/projected/fcafe656-9fdf-4609-9b2b-eae73b8e4677-kube-api-access-67lnj\") pod \"cert-manager-webhook-6486ffd997-fsztt\" (UID: \"fcafe656-9fdf-4609-9b2b-eae73b8e4677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.336237 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.336208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcafe656-9fdf-4609-9b2b-eae73b8e4677-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fsztt\" (UID: \"fcafe656-9fdf-4609-9b2b-eae73b8e4677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.336237 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.336237 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lnj\" (UniqueName: \"kubernetes.io/projected/fcafe656-9fdf-4609-9b2b-eae73b8e4677-kube-api-access-67lnj\") pod \"cert-manager-webhook-6486ffd997-fsztt\" (UID: \"fcafe656-9fdf-4609-9b2b-eae73b8e4677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.344301 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.344276 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lnj\" (UniqueName: \"kubernetes.io/projected/fcafe656-9fdf-4609-9b2b-eae73b8e4677-kube-api-access-67lnj\") pod \"cert-manager-webhook-6486ffd997-fsztt\" (UID: \"fcafe656-9fdf-4609-9b2b-eae73b8e4677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.344399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.344318 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fcafe656-9fdf-4609-9b2b-eae73b8e4677-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fsztt\" (UID: \"fcafe656-9fdf-4609-9b2b-eae73b8e4677\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.462963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.462931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:24.601393 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.601361 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fsztt"] Jul 24 13:13:24.604720 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:13:24.604692 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcafe656_9fdf_4609_9b2b_eae73b8e4677.slice/crio-d9f90915c183ed1d260cc1ee54d1054409806c9352d255594e7f4306c4c58e91 WatchSource:0}: Error finding container d9f90915c183ed1d260cc1ee54d1054409806c9352d255594e7f4306c4c58e91: Status 404 returned error can't find the container with id d9f90915c183ed1d260cc1ee54d1054409806c9352d255594e7f4306c4c58e91 Jul 24 13:13:24.866621 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:24.866529 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" event={"ID":"fcafe656-9fdf-4609-9b2b-eae73b8e4677","Type":"ContainerStarted","Data":"d9f90915c183ed1d260cc1ee54d1054409806c9352d255594e7f4306c4c58e91"} Jul 24 13:13:25.844958 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.844924 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt"] Jul 24 13:13:25.847267 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.847246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:25.849524 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.849504 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dzqnd\"" Jul 24 13:13:25.858364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.858344 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt"] Jul 24 13:13:25.874642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.874609 2562 generic.go:356] "Generic (PLEG): container finished" podID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerID="222c92a234b76b20d925a66c1327d16ee3ad26a9cbf76684b3c0429951490346" exitCode=0 Jul 24 13:13:25.874981 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.874676 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" event={"ID":"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9","Type":"ContainerDied","Data":"222c92a234b76b20d925a66c1327d16ee3ad26a9cbf76684b3c0429951490346"} Jul 24 13:13:25.946530 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.946499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e65d6061-3ace-4e7a-88a1-0cc43bf043de-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7zjgt\" (UID: \"e65d6061-3ace-4e7a-88a1-0cc43bf043de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:25.946679 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:25.946624 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrjd5\" (UniqueName: \"kubernetes.io/projected/e65d6061-3ace-4e7a-88a1-0cc43bf043de-kube-api-access-wrjd5\") pod \"cert-manager-cainjector-559cdd8c87-7zjgt\" (UID: \"e65d6061-3ace-4e7a-88a1-0cc43bf043de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:26.047845 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.047813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e65d6061-3ace-4e7a-88a1-0cc43bf043de-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7zjgt\" (UID: \"e65d6061-3ace-4e7a-88a1-0cc43bf043de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:26.047969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.047906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrjd5\" (UniqueName: \"kubernetes.io/projected/e65d6061-3ace-4e7a-88a1-0cc43bf043de-kube-api-access-wrjd5\") pod \"cert-manager-cainjector-559cdd8c87-7zjgt\" (UID: \"e65d6061-3ace-4e7a-88a1-0cc43bf043de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:26.057771 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.057748 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e65d6061-3ace-4e7a-88a1-0cc43bf043de-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7zjgt\" (UID: \"e65d6061-3ace-4e7a-88a1-0cc43bf043de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:26.058223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.058203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrjd5\" (UniqueName: \"kubernetes.io/projected/e65d6061-3ace-4e7a-88a1-0cc43bf043de-kube-api-access-wrjd5\") pod \"cert-manager-cainjector-559cdd8c87-7zjgt\" (UID: \"e65d6061-3ace-4e7a-88a1-0cc43bf043de\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:26.156876 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.156848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" Jul 24 13:13:26.313475 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.313444 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt"] Jul 24 13:13:26.315913 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:13:26.315884 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode65d6061_3ace_4e7a_88a1_0cc43bf043de.slice/crio-004053c8f36f41e18df7c900ceca41dc0a817f23b34fca65a3d1253f73481bba WatchSource:0}: Error finding container 004053c8f36f41e18df7c900ceca41dc0a817f23b34fca65a3d1253f73481bba: Status 404 returned error can't find the container with id 004053c8f36f41e18df7c900ceca41dc0a817f23b34fca65a3d1253f73481bba Jul 24 13:13:26.880429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.880391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" event={"ID":"e65d6061-3ace-4e7a-88a1-0cc43bf043de","Type":"ContainerStarted","Data":"004053c8f36f41e18df7c900ceca41dc0a817f23b34fca65a3d1253f73481bba"} Jul 24 13:13:26.882693 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.882662 2562 generic.go:356] "Generic (PLEG): container finished" podID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerID="f95e7878ce8ee1f181edba84d7b013c6e24998120240a11e964acc36d44d6dc0" exitCode=0 Jul 24 13:13:26.882814 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:26.882702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" event={"ID":"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9","Type":"ContainerDied","Data":"f95e7878ce8ee1f181edba84d7b013c6e24998120240a11e964acc36d44d6dc0"} Jul 24 13:13:27.888911 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:27.888868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" event={"ID":"e65d6061-3ace-4e7a-88a1-0cc43bf043de","Type":"ContainerStarted","Data":"225d38e0084ca3e57333ada60f8339cca67693534fb4788c9ebc986e4b2d31a3"} Jul 24 13:13:27.896356 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:27.896328 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" event={"ID":"fcafe656-9fdf-4609-9b2b-eae73b8e4677","Type":"ContainerStarted","Data":"4214758c4331b9c4c00fbde55aeccab42804dc79146a3225ce13ffd8df197731"} Jul 24 13:13:27.896473 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:27.896371 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:27.907428 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:27.907380 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7zjgt" podStartSLOduration=1.540748005 podStartE2EDuration="2.907364745s" totalImagesPullingTime="1.36661674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:13:27.906860823 +0000 UTC m=+234.506106035" watchObservedRunningTime="2026-07-24 13:13:27.907364745 +0000 UTC m=+234.506609933" Jul 24 13:13:27.927810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:27.927739 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" podStartSLOduration=0.850011884 podStartE2EDuration="3.927715402s" totalImagesPullingTime="3.077703518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:13:27.925126867 +0000 UTC m=+234.524372080" watchObservedRunningTime="2026-07-24 13:13:27.927715402 +0000 UTC m=+234.526960614" Jul 24 13:13:28.032816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.032791 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:28.062123 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.062094 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x6rd\" (UniqueName: \"kubernetes.io/projected/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-kube-api-access-7x6rd\") pod \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " Jul 24 13:13:28.062246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.062175 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-bundle\") pod \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " Jul 24 13:13:28.062246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.062216 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-util\") pod \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\" (UID: \"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9\") " Jul 24 13:13:28.062585 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.062541 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-bundle" (OuterVolumeSpecName: "bundle") pod "d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" (UID: "d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:13:28.064109 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.064083 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-kube-api-access-7x6rd" (OuterVolumeSpecName: "kube-api-access-7x6rd") pod "d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" (UID: "d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9"). InnerVolumeSpecName "kube-api-access-7x6rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:13:28.068027 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.067999 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-util" (OuterVolumeSpecName: "util") pod "d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" (UID: "d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:13:28.167267 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.167228 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x6rd\" (UniqueName: \"kubernetes.io/projected/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-kube-api-access-7x6rd\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:28.167433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.167417 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:28.167507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.167434 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:28.896427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.896398 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" Jul 24 13:13:28.896956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.896920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8xl44" event={"ID":"d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9","Type":"ContainerDied","Data":"6950d55b28d94bbf69aadcc4bb69128a163f22d15fbe69be10e4a652ea2b68f3"} Jul 24 13:13:28.897067 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:28.896976 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6950d55b28d94bbf69aadcc4bb69128a163f22d15fbe69be10e4a652ea2b68f3" Jul 24 13:13:33.898815 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:33.898790 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fsztt" Jul 24 13:13:40.311955 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.311921 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn"] Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312705 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="util" Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312735 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="util" Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312772 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="extract" Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312793 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="extract" Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312823 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="pull" Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312833 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="pull" Jul 24 13:13:40.314997 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.312996 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5cd27cd-04fa-4db6-bcce-ad1c37cc88a9" containerName="extract" Jul 24 13:13:40.317659 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.317633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.320125 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.320104 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 13:13:40.320239 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.320149 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 13:13:40.320632 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.320618 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wxfsb\"" Jul 24 13:13:40.325144 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.325126 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn"] Jul 24 13:13:40.353593 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.353571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.353699 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.353612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs5wr\" (UniqueName: \"kubernetes.io/projected/a2429431-e056-48d1-9cc1-18240894292c-kube-api-access-bs5wr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.353699 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.353647 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.454288 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.454255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.454431 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.454297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs5wr\" (UniqueName: \"kubernetes.io/projected/a2429431-e056-48d1-9cc1-18240894292c-kube-api-access-bs5wr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.454431 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.454334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.454737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.454712 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.454784 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.454744 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.462858 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.462830 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs5wr\" (UniqueName: \"kubernetes.io/projected/a2429431-e056-48d1-9cc1-18240894292c-kube-api-access-bs5wr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.628107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.628034 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:40.747238 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.747216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn"] Jul 24 13:13:40.749305 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:13:40.749280 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2429431_e056_48d1_9cc1_18240894292c.slice/crio-11999119d48eeff5f78d2ee8992cbf84c308a9f24e7c9dfec5c3bb184178687a WatchSource:0}: Error finding container 11999119d48eeff5f78d2ee8992cbf84c308a9f24e7c9dfec5c3bb184178687a: Status 404 returned error can't find the container with id 11999119d48eeff5f78d2ee8992cbf84c308a9f24e7c9dfec5c3bb184178687a Jul 24 13:13:40.944774 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.944744 2562 generic.go:356] "Generic (PLEG): container finished" podID="a2429431-e056-48d1-9cc1-18240894292c" containerID="cf655f73821af3eaaec0305d4ff411507165fe50373ad2cf5197dc0ee779f531" exitCode=0 Jul 24 13:13:40.944927 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.944831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" event={"ID":"a2429431-e056-48d1-9cc1-18240894292c","Type":"ContainerDied","Data":"cf655f73821af3eaaec0305d4ff411507165fe50373ad2cf5197dc0ee779f531"} Jul 24 13:13:40.944927 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:40.944885 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" event={"ID":"a2429431-e056-48d1-9cc1-18240894292c","Type":"ContainerStarted","Data":"11999119d48eeff5f78d2ee8992cbf84c308a9f24e7c9dfec5c3bb184178687a"} Jul 24 13:13:41.953040 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:41.953010 2562 generic.go:356] "Generic (PLEG): container finished" podID="a2429431-e056-48d1-9cc1-18240894292c" containerID="31a41789f85b21f08d97b1d066d10fbc6e9989ad97bc070517a291473b5b1bc7" exitCode=0 Jul 24 13:13:41.953372 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:41.953087 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" event={"ID":"a2429431-e056-48d1-9cc1-18240894292c","Type":"ContainerDied","Data":"31a41789f85b21f08d97b1d066d10fbc6e9989ad97bc070517a291473b5b1bc7"} Jul 24 13:13:42.959184 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:42.959151 2562 generic.go:356] "Generic (PLEG): container finished" podID="a2429431-e056-48d1-9cc1-18240894292c" containerID="c60342ede430575a61a9a4581b19613e51b55fe5dd586d20312406cc57f9437c" exitCode=0 Jul 24 13:13:42.959522 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:42.959211 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" event={"ID":"a2429431-e056-48d1-9cc1-18240894292c","Type":"ContainerDied","Data":"c60342ede430575a61a9a4581b19613e51b55fe5dd586d20312406cc57f9437c"} Jul 24 13:13:44.100114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.100092 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:44.183393 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.183362 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs5wr\" (UniqueName: \"kubernetes.io/projected/a2429431-e056-48d1-9cc1-18240894292c-kube-api-access-bs5wr\") pod \"a2429431-e056-48d1-9cc1-18240894292c\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " Jul 24 13:13:44.183529 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.183403 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-util\") pod \"a2429431-e056-48d1-9cc1-18240894292c\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " Jul 24 13:13:44.183529 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.183467 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-bundle\") pod \"a2429431-e056-48d1-9cc1-18240894292c\" (UID: \"a2429431-e056-48d1-9cc1-18240894292c\") " Jul 24 13:13:44.184329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.184299 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-bundle" (OuterVolumeSpecName: "bundle") pod "a2429431-e056-48d1-9cc1-18240894292c" (UID: "a2429431-e056-48d1-9cc1-18240894292c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:13:44.185365 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.185336 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2429431-e056-48d1-9cc1-18240894292c-kube-api-access-bs5wr" (OuterVolumeSpecName: "kube-api-access-bs5wr") pod "a2429431-e056-48d1-9cc1-18240894292c" (UID: "a2429431-e056-48d1-9cc1-18240894292c"). InnerVolumeSpecName "kube-api-access-bs5wr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:13:44.189401 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.189361 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-util" (OuterVolumeSpecName: "util") pod "a2429431-e056-48d1-9cc1-18240894292c" (UID: "a2429431-e056-48d1-9cc1-18240894292c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:13:44.284332 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.284251 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:44.284332 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.284278 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs5wr\" (UniqueName: \"kubernetes.io/projected/a2429431-e056-48d1-9cc1-18240894292c-kube-api-access-bs5wr\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:44.284332 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.284288 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2429431-e056-48d1-9cc1-18240894292c-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:13:44.969951 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.969913 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" event={"ID":"a2429431-e056-48d1-9cc1-18240894292c","Type":"ContainerDied","Data":"11999119d48eeff5f78d2ee8992cbf84c308a9f24e7c9dfec5c3bb184178687a"} Jul 24 13:13:44.970111 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.969968 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgmkxn" Jul 24 13:13:44.970111 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:44.969974 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11999119d48eeff5f78d2ee8992cbf84c308a9f24e7c9dfec5c3bb184178687a" Jul 24 13:13:55.092059 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092024 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk"] Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092424 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="extract" Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092442 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="extract" Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092466 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="pull" Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092474 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="pull" Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092485 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="util" Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092493 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="util" Jul 24 13:13:55.092691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.092569 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2429431-e056-48d1-9cc1-18240894292c" containerName="extract" Jul 24 13:13:55.095392 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.095373 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.097950 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.097869 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 13:13:55.097950 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.097894 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 13:13:55.097950 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.097923 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-84bfr\"" Jul 24 13:13:55.097950 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.097955 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 13:13:55.098218 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.098001 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 13:13:55.117467 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.117442 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk"] Jul 24 13:13:55.162897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.162871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2a56fe7-4957-4e9e-91df-db4c570b7535-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.163005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.162907 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2a56fe7-4957-4e9e-91df-db4c570b7535-manifests\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.163005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.162934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzwl4\" (UniqueName: \"kubernetes.io/projected/c2a56fe7-4957-4e9e-91df-db4c570b7535-kube-api-access-qzwl4\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.163121 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.163023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2a56fe7-4957-4e9e-91df-db4c570b7535-tmp\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.163121 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.163062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2a56fe7-4957-4e9e-91df-db4c570b7535-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.263929 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.263901 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2a56fe7-4957-4e9e-91df-db4c570b7535-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.263929 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.263932 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2a56fe7-4957-4e9e-91df-db4c570b7535-manifests\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.264134 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.263953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzwl4\" (UniqueName: \"kubernetes.io/projected/c2a56fe7-4957-4e9e-91df-db4c570b7535-kube-api-access-qzwl4\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.264134 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.264070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2a56fe7-4957-4e9e-91df-db4c570b7535-tmp\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.264134 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.264110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2a56fe7-4957-4e9e-91df-db4c570b7535-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.264377 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.264360 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2a56fe7-4957-4e9e-91df-db4c570b7535-manifests\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.266175 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.266155 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2a56fe7-4957-4e9e-91df-db4c570b7535-tmp\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.266409 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.266391 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2a56fe7-4957-4e9e-91df-db4c570b7535-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.266461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.266425 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2a56fe7-4957-4e9e-91df-db4c570b7535-webhook-cert\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.272184 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.272161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzwl4\" (UniqueName: \"kubernetes.io/projected/c2a56fe7-4957-4e9e-91df-db4c570b7535-kube-api-access-qzwl4\") pod \"opendatahub-operator-controller-manager-dbfd75c57-gk7vk\" (UID: \"c2a56fe7-4957-4e9e-91df-db4c570b7535\") " pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.405589 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.405455 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:13:55.538766 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:55.538736 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk"] Jul 24 13:13:55.541888 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:13:55.541864 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a56fe7_4957_4e9e_91df_db4c570b7535.slice/crio-34240edb5b992ed873730590445b13462e203eae04797ba0cf3d89ade587fd3e WatchSource:0}: Error finding container 34240edb5b992ed873730590445b13462e203eae04797ba0cf3d89ade587fd3e: Status 404 returned error can't find the container with id 34240edb5b992ed873730590445b13462e203eae04797ba0cf3d89ade587fd3e Jul 24 13:13:56.016117 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:56.016080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" event={"ID":"c2a56fe7-4957-4e9e-91df-db4c570b7535","Type":"ContainerStarted","Data":"34240edb5b992ed873730590445b13462e203eae04797ba0cf3d89ade587fd3e"} Jul 24 13:13:59.031853 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:13:59.031817 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" event={"ID":"c2a56fe7-4957-4e9e-91df-db4c570b7535","Type":"ContainerStarted","Data":"1eefe02932e9da7df29054cd80c3774044926aebf226502d1705bc8f4a05376b"} Jul 24 13:14:00.037115 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:00.037083 2562 generic.go:356] "Generic (PLEG): container finished" podID="c2a56fe7-4957-4e9e-91df-db4c570b7535" containerID="1eefe02932e9da7df29054cd80c3774044926aebf226502d1705bc8f4a05376b" exitCode=0 Jul 24 13:14:00.037493 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:00.037137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" event={"ID":"c2a56fe7-4957-4e9e-91df-db4c570b7535","Type":"ContainerDied","Data":"1eefe02932e9da7df29054cd80c3774044926aebf226502d1705bc8f4a05376b"} Jul 24 13:14:01.043849 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:01.043815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" event={"ID":"c2a56fe7-4957-4e9e-91df-db4c570b7535","Type":"ContainerStarted","Data":"bcbf6300ae7cfa693ddda9e2989e4e2621b082e6c82af684daf49a860b40e60d"} Jul 24 13:14:01.044244 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:01.043934 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:14:01.070567 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:01.070507 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" podStartSLOduration=1.2967850859999999 podStartE2EDuration="6.07049267s" totalImagesPullingTime="3.705403188s" totalInitContainerRuntime="1.068304396s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:13:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:01.069148689 +0000 UTC m=+267.668393900" watchObservedRunningTime="2026-07-24 13:14:01.07049267 +0000 UTC m=+267.669737881" Jul 24 13:14:12.050305 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:12.050275 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dbfd75c57-gk7vk" Jul 24 13:14:13.073156 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.073119 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql"] Jul 24 13:14:13.081330 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.081310 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.084620 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084591 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 13:14:13.084742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084641 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 13:14:13.084742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084656 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 13:14:13.084742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084725 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql"] Jul 24 13:14:13.084892 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084605 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 13:14:13.084892 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084865 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 13:14:13.084892 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.084883 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rcphn\"" Jul 24 13:14:13.211163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.211131 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ecea6c0-8d5c-4903-8070-aad08d9583c7-metrics-cert\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.211339 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.211170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl8f7\" (UniqueName: \"kubernetes.io/projected/8ecea6c0-8d5c-4903-8070-aad08d9583c7-kube-api-access-bl8f7\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.211339 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.211195 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8ecea6c0-8d5c-4903-8070-aad08d9583c7-manager-config\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.211339 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.211266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ecea6c0-8d5c-4903-8070-aad08d9583c7-cert\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.312000 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.311967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ecea6c0-8d5c-4903-8070-aad08d9583c7-cert\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.312155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.312011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ecea6c0-8d5c-4903-8070-aad08d9583c7-metrics-cert\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.312155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.312125 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl8f7\" (UniqueName: \"kubernetes.io/projected/8ecea6c0-8d5c-4903-8070-aad08d9583c7-kube-api-access-bl8f7\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.312227 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.312164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8ecea6c0-8d5c-4903-8070-aad08d9583c7-manager-config\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.312802 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.312781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8ecea6c0-8d5c-4903-8070-aad08d9583c7-manager-config\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.314429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.314407 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8ecea6c0-8d5c-4903-8070-aad08d9583c7-metrics-cert\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.314514 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.314469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ecea6c0-8d5c-4903-8070-aad08d9583c7-cert\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.320784 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.320764 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl8f7\" (UniqueName: \"kubernetes.io/projected/8ecea6c0-8d5c-4903-8070-aad08d9583c7-kube-api-access-bl8f7\") pod \"lws-controller-manager-5d6cbddddd-v4bql\" (UID: \"8ecea6c0-8d5c-4903-8070-aad08d9583c7\") " pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.391785 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.391719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:13.518315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:13.518268 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql"] Jul 24 13:14:13.520611 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:14:13.520581 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ecea6c0_8d5c_4903_8070_aad08d9583c7.slice/crio-e424abe70a1b01a4dc297cb31a41a9cc059a7f501e325c969e2db0b2c1d4181b WatchSource:0}: Error finding container e424abe70a1b01a4dc297cb31a41a9cc059a7f501e325c969e2db0b2c1d4181b: Status 404 returned error can't find the container with id e424abe70a1b01a4dc297cb31a41a9cc059a7f501e325c969e2db0b2c1d4181b Jul 24 13:14:14.097825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:14.097793 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" event={"ID":"8ecea6c0-8d5c-4903-8070-aad08d9583c7","Type":"ContainerStarted","Data":"e424abe70a1b01a4dc297cb31a41a9cc059a7f501e325c969e2db0b2c1d4181b"} Jul 24 13:14:16.109498 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:16.109464 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" event={"ID":"8ecea6c0-8d5c-4903-8070-aad08d9583c7","Type":"ContainerStarted","Data":"5f4af1cfd221c9d8a41f99714ee52577fbce7ed139425ba46f6fc81f84ee95b6"} Jul 24 13:14:16.109912 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:16.109611 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:16.138304 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:16.138251 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" podStartSLOduration=1.4494612359999999 podStartE2EDuration="3.138235038s" totalImagesPullingTime="1.688773802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:16.136921894 +0000 UTC m=+282.736167106" watchObservedRunningTime="2026-07-24 13:14:16.138235038 +0000 UTC m=+282.737480249" Jul 24 13:14:22.739114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.739076 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh"] Jul 24 13:14:22.743852 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.743830 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.747500 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.747478 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 13:14:22.747640 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.747531 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gfvhz\"" Jul 24 13:14:22.748138 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.748121 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 13:14:22.748356 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.748340 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 13:14:22.765399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.765378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh"] Jul 24 13:14:22.871658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.871628 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt"] Jul 24 13:14:22.874769 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.874751 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.877494 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.877468 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 24 13:14:22.877650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.877623 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 24 13:14:22.877746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.877671 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 24 13:14:22.877815 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.877799 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-db6c7\"" Jul 24 13:14:22.887014 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.886992 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt"] Jul 24 13:14:22.891092 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891180 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891112 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9589e305-2fff-4be8-b990-a7e1debb86da-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891180 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9589e305-2fff-4be8-b990-a7e1debb86da-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891251 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls5tl\" (UniqueName: \"kubernetes.io/projected/9589e305-2fff-4be8-b990-a7e1debb86da-kube-api-access-ls5tl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891285 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891270 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891318 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891348 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9589e305-2fff-4be8-b990-a7e1debb86da-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891383 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.891413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.891390 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992189 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klnx9\" (UniqueName: \"kubernetes.io/projected/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-kube-api-access-klnx9\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992189 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls5tl\" (UniqueName: \"kubernetes.io/projected/9589e305-2fff-4be8-b990-a7e1debb86da-kube-api-access-ls5tl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992189 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992244 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9589e305-2fff-4be8-b990-a7e1debb86da-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992384 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9589e305-2fff-4be8-b990-a7e1debb86da-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9589e305-2fff-4be8-b990-a7e1debb86da-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:22.992737 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992675 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.993123 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.992830 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.993177 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.993115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.993177 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.993156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.993690 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.993667 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9589e305-2fff-4be8-b990-a7e1debb86da-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.995049 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.995030 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9589e305-2fff-4be8-b990-a7e1debb86da-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:22.995226 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:22.995211 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9589e305-2fff-4be8-b990-a7e1debb86da-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:23.000996 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.000975 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls5tl\" (UniqueName: \"kubernetes.io/projected/9589e305-2fff-4be8-b990-a7e1debb86da-kube-api-access-ls5tl\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:23.001439 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.001418 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9589e305-2fff-4be8-b990-a7e1debb86da-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh\" (UID: \"9589e305-2fff-4be8-b990-a7e1debb86da\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:23.053606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.053563 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:23.094133 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.094307 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.094307 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094191 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.094307 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.094307 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klnx9\" (UniqueName: \"kubernetes.io/projected/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-kube-api-access-klnx9\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.094307 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.094598 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.094346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.095464 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.095420 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.097641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.097593 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.097641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.097622 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.098282 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.097909 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.098282 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.098232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.106746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.106707 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klnx9\" (UniqueName: \"kubernetes.io/projected/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-kube-api-access-klnx9\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.106746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.106710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8bmrt\" (UID: \"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.183867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.183837 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:23.215565 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.215513 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh"] Jul 24 13:14:23.219521 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:14:23.219495 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9589e305_2fff_4be8_b990_a7e1debb86da.slice/crio-0b08e1b6ecf8f69019d910180d0f9f86d66c7c28cc79627493b6e1ba31087d30 WatchSource:0}: Error finding container 0b08e1b6ecf8f69019d910180d0f9f86d66c7c28cc79627493b6e1ba31087d30: Status 404 returned error can't find the container with id 0b08e1b6ecf8f69019d910180d0f9f86d66c7c28cc79627493b6e1ba31087d30 Jul 24 13:14:23.331382 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:23.331244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt"] Jul 24 13:14:23.334431 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:14:23.334403 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9462b9c_63dd_4327_8bcb_f6fe0b4b24ef.slice/crio-4ec3019877c5ab64b9383464fcd753d7ea0e854ae27da06fad3b197c005cf20f WatchSource:0}: Error finding container 4ec3019877c5ab64b9383464fcd753d7ea0e854ae27da06fad3b197c005cf20f: Status 404 returned error can't find the container with id 4ec3019877c5ab64b9383464fcd753d7ea0e854ae27da06fad3b197c005cf20f Jul 24 13:14:24.150891 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:24.150852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" event={"ID":"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef","Type":"ContainerStarted","Data":"4ec3019877c5ab64b9383464fcd753d7ea0e854ae27da06fad3b197c005cf20f"} Jul 24 13:14:24.152281 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:24.152248 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" event={"ID":"9589e305-2fff-4be8-b990-a7e1debb86da","Type":"ContainerStarted","Data":"0b08e1b6ecf8f69019d910180d0f9f86d66c7c28cc79627493b6e1ba31087d30"} Jul 24 13:14:27.019730 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.019691 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:27.020120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.019775 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:27.023944 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.023914 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:27.024005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.023976 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:27.024040 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.024005 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:14:27.116643 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.116606 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5d6cbddddd-v4bql" Jul 24 13:14:27.172223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.172081 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" event={"ID":"c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef","Type":"ContainerStarted","Data":"b77be24b3d5f80268c27a2249c1fc5e67793abd959814cfe5c35a9f2ed2faae7"} Jul 24 13:14:27.172223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.172164 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:27.174232 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.174204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" event={"ID":"9589e305-2fff-4be8-b990-a7e1debb86da","Type":"ContainerStarted","Data":"58742accaefdaacf5b5daae620599e98a2d6ed5b3c84eb09e12fd28d657a798a"} Jul 24 13:14:27.198840 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.198757 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" podStartSLOduration=1.51600598 podStartE2EDuration="5.198740141s" totalImagesPullingTime="3.682734161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:27.197885124 +0000 UTC m=+293.797130349" watchObservedRunningTime="2026-07-24 13:14:27.198740141 +0000 UTC m=+293.797985330" Jul 24 13:14:27.224353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:27.224308 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" podStartSLOduration=1.422620542 podStartE2EDuration="5.224291485s" totalImagesPullingTime="3.801670943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:14:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:14:27.222883309 +0000 UTC m=+293.822128521" watchObservedRunningTime="2026-07-24 13:14:27.224291485 +0000 UTC m=+293.823536698" Jul 24 13:14:28.054716 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:28.054684 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:28.059287 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:28.059262 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:28.178589 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:28.178562 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:28.179525 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:28.179498 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh" Jul 24 13:14:28.180136 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:28.180117 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8bmrt" Jul 24 13:14:33.715804 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:14:33.715767 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 13:15:28.932378 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:28.932299 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn"] Jul 24 13:15:28.935793 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:28.935777 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:28.938724 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:28.938706 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v9v7d\"" Jul 24 13:15:28.938863 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:28.938845 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:15:28.939465 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:28.939446 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:15:28.946429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:28.946407 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn"] Jul 24 13:15:29.023901 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.023873 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4lzb\" (UniqueName: \"kubernetes.io/projected/760249a8-7a07-496c-a5a0-c4f147034025-kube-api-access-b4lzb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.023901 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.023905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.024089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.023923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.124758 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.124730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4lzb\" (UniqueName: \"kubernetes.io/projected/760249a8-7a07-496c-a5a0-c4f147034025-kube-api-access-b4lzb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.124758 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.124761 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.124931 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.124786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.125128 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.125110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.125167 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.125150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.133161 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.133131 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4lzb\" (UniqueName: \"kubernetes.io/projected/760249a8-7a07-496c-a5a0-c4f147034025-kube-api-access-b4lzb\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.245357 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.245290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:29.376974 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.376951 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn"] Jul 24 13:15:29.380291 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:15:29.380254 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760249a8_7a07_496c_a5a0_c4f147034025.slice/crio-a9064638a14516b777494844a5f71f16c52fcef877cd83609a568361caad4ba9 WatchSource:0}: Error finding container a9064638a14516b777494844a5f71f16c52fcef877cd83609a568361caad4ba9: Status 404 returned error can't find the container with id a9064638a14516b777494844a5f71f16c52fcef877cd83609a568361caad4ba9 Jul 24 13:15:29.449958 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.449932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" event={"ID":"760249a8-7a07-496c-a5a0-c4f147034025","Type":"ContainerStarted","Data":"915bf38c113b0b8725c3439486b9380de8661924494813d48dc72558aea14557"} Jul 24 13:15:29.450072 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.449980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" event={"ID":"760249a8-7a07-496c-a5a0-c4f147034025","Type":"ContainerStarted","Data":"a9064638a14516b777494844a5f71f16c52fcef877cd83609a568361caad4ba9"} Jul 24 13:15:29.704117 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.704085 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f"] Jul 24 13:15:29.707494 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.707472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.716673 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.716653 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f"] Jul 24 13:15:29.831264 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.831236 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.831438 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.831274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c62t8\" (UniqueName: \"kubernetes.io/projected/8d073917-4713-459e-9960-b177cacae466-kube-api-access-c62t8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.831438 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.831304 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.931783 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.931753 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.931928 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.931798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c62t8\" (UniqueName: \"kubernetes.io/projected/8d073917-4713-459e-9960-b177cacae466-kube-api-access-c62t8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.931928 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.931840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.932215 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.932196 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.932253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.932206 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:29.941267 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:29.941236 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c62t8\" (UniqueName: \"kubernetes.io/projected/8d073917-4713-459e-9960-b177cacae466-kube-api-access-c62t8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:30.016508 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.016438 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:30.142643 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.142568 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f"] Jul 24 13:15:30.144830 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:15:30.144801 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d073917_4713_459e_9960_b177cacae466.slice/crio-4551d87d63e2ed6a504b369f690d1d057fe5c4ff4e9005c3b8ccc83296fc98d7 WatchSource:0}: Error finding container 4551d87d63e2ed6a504b369f690d1d057fe5c4ff4e9005c3b8ccc83296fc98d7: Status 404 returned error can't find the container with id 4551d87d63e2ed6a504b369f690d1d057fe5c4ff4e9005c3b8ccc83296fc98d7 Jul 24 13:15:30.313450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.313385 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc"] Jul 24 13:15:30.316816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.316802 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.326212 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.326189 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc"] Jul 24 13:15:30.435931 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.435903 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfv64\" (UniqueName: \"kubernetes.io/projected/9806a155-10e3-4b57-a897-d9443c7b68a4-kube-api-access-vfv64\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.436084 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.435983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.436084 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.436013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.455947 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.455919 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d073917-4713-459e-9960-b177cacae466" containerID="c094a5aff5c02ae22eccb3fc76f470a80fd4a49e0addc0e743b315f820c77ecc" exitCode=0 Jul 24 13:15:30.456069 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.456000 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" event={"ID":"8d073917-4713-459e-9960-b177cacae466","Type":"ContainerDied","Data":"c094a5aff5c02ae22eccb3fc76f470a80fd4a49e0addc0e743b315f820c77ecc"} Jul 24 13:15:30.456069 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.456041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" event={"ID":"8d073917-4713-459e-9960-b177cacae466","Type":"ContainerStarted","Data":"4551d87d63e2ed6a504b369f690d1d057fe5c4ff4e9005c3b8ccc83296fc98d7"} Jul 24 13:15:30.456527 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.456510 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:15:30.457627 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.457601 2562 generic.go:356] "Generic (PLEG): container finished" podID="760249a8-7a07-496c-a5a0-c4f147034025" containerID="915bf38c113b0b8725c3439486b9380de8661924494813d48dc72558aea14557" exitCode=0 Jul 24 13:15:30.457711 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.457686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" event={"ID":"760249a8-7a07-496c-a5a0-c4f147034025","Type":"ContainerDied","Data":"915bf38c113b0b8725c3439486b9380de8661924494813d48dc72558aea14557"} Jul 24 13:15:30.537081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.537048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.537081 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.537087 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.537282 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.537119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfv64\" (UniqueName: \"kubernetes.io/projected/9806a155-10e3-4b57-a897-d9443c7b68a4-kube-api-access-vfv64\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.537518 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.537498 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.537608 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.537591 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.546601 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.546573 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfv64\" (UniqueName: \"kubernetes.io/projected/9806a155-10e3-4b57-a897-d9443c7b68a4-kube-api-access-vfv64\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.626403 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.626325 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:30.755194 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.755144 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc"] Jul 24 13:15:30.757582 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:15:30.757539 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9806a155_10e3_4b57_a897_d9443c7b68a4.slice/crio-8c62fd8234a433877af69b98f0657e004c25bae9067a7e3ce61d1075acab3c1c WatchSource:0}: Error finding container 8c62fd8234a433877af69b98f0657e004c25bae9067a7e3ce61d1075acab3c1c: Status 404 returned error can't find the container with id 8c62fd8234a433877af69b98f0657e004c25bae9067a7e3ce61d1075acab3c1c Jul 24 13:15:30.955140 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.955107 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p"] Jul 24 13:15:30.958433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.958417 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:30.971681 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:30.971656 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p"] Jul 24 13:15:31.141531 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.141499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.141666 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.141536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.141732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.141685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vlbz\" (UniqueName: \"kubernetes.io/projected/98f94866-c253-4b6b-8c0c-878220778359-kube-api-access-7vlbz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.242207 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.242131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vlbz\" (UniqueName: \"kubernetes.io/projected/98f94866-c253-4b6b-8c0c-878220778359-kube-api-access-7vlbz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.242341 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.242267 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.242341 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.242287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.242638 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.242616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.242638 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.242628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.253664 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.253642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vlbz\" (UniqueName: \"kubernetes.io/projected/98f94866-c253-4b6b-8c0c-878220778359-kube-api-access-7vlbz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.267303 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.267282 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:31.464248 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.464218 2562 generic.go:356] "Generic (PLEG): container finished" podID="760249a8-7a07-496c-a5a0-c4f147034025" containerID="f6945f771ca915d11028226423688806c6d8a69dcc7d92cd47659ed476ae6353" exitCode=0 Jul 24 13:15:31.464352 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.464298 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" event={"ID":"760249a8-7a07-496c-a5a0-c4f147034025","Type":"ContainerDied","Data":"f6945f771ca915d11028226423688806c6d8a69dcc7d92cd47659ed476ae6353"} Jul 24 13:15:31.465990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.465970 2562 generic.go:356] "Generic (PLEG): container finished" podID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerID="926adbc8849778447b73d0dabe95804e6508b352543298bfb912909b576bc612" exitCode=0 Jul 24 13:15:31.466073 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.466050 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" event={"ID":"9806a155-10e3-4b57-a897-d9443c7b68a4","Type":"ContainerDied","Data":"926adbc8849778447b73d0dabe95804e6508b352543298bfb912909b576bc612"} Jul 24 13:15:31.466116 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.466106 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" event={"ID":"9806a155-10e3-4b57-a897-d9443c7b68a4","Type":"ContainerStarted","Data":"8c62fd8234a433877af69b98f0657e004c25bae9067a7e3ce61d1075acab3c1c"} Jul 24 13:15:31.483111 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:31.482575 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p"] Jul 24 13:15:31.486016 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:15:31.485996 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f94866_c253_4b6b_8c0c_878220778359.slice/crio-d0b47512392d6591ab047abc2fcb95ad472743964d9cb639e7ada435b301f95f WatchSource:0}: Error finding container d0b47512392d6591ab047abc2fcb95ad472743964d9cb639e7ada435b301f95f: Status 404 returned error can't find the container with id d0b47512392d6591ab047abc2fcb95ad472743964d9cb639e7ada435b301f95f Jul 24 13:15:32.472016 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.471987 2562 generic.go:356] "Generic (PLEG): container finished" podID="98f94866-c253-4b6b-8c0c-878220778359" containerID="9f158b9f190e59611fe7a2d97a267ff30a15bb7c0c27e6beaa412b87d6c5c49b" exitCode=0 Jul 24 13:15:32.472413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.472072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" event={"ID":"98f94866-c253-4b6b-8c0c-878220778359","Type":"ContainerDied","Data":"9f158b9f190e59611fe7a2d97a267ff30a15bb7c0c27e6beaa412b87d6c5c49b"} Jul 24 13:15:32.472413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.472140 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" event={"ID":"98f94866-c253-4b6b-8c0c-878220778359","Type":"ContainerStarted","Data":"d0b47512392d6591ab047abc2fcb95ad472743964d9cb639e7ada435b301f95f"} Jul 24 13:15:32.474338 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.474256 2562 generic.go:356] "Generic (PLEG): container finished" podID="760249a8-7a07-496c-a5a0-c4f147034025" containerID="b550280831a015dc21d579c21f28ad63ec0a7f4324059e1e7a142b13a45d2a3b" exitCode=0 Jul 24 13:15:32.474413 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.474352 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" event={"ID":"760249a8-7a07-496c-a5a0-c4f147034025","Type":"ContainerDied","Data":"b550280831a015dc21d579c21f28ad63ec0a7f4324059e1e7a142b13a45d2a3b"} Jul 24 13:15:32.476163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.476143 2562 generic.go:356] "Generic (PLEG): container finished" podID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerID="148e8a7c802b9f7e725a6067d7be139e25267b642fbe8848e36b11ba9bd43f69" exitCode=0 Jul 24 13:15:32.476277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.476229 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" event={"ID":"9806a155-10e3-4b57-a897-d9443c7b68a4","Type":"ContainerDied","Data":"148e8a7c802b9f7e725a6067d7be139e25267b642fbe8848e36b11ba9bd43f69"} Jul 24 13:15:32.478119 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.478095 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d073917-4713-459e-9960-b177cacae466" containerID="98b56430abea48e77c2cab88eb22bdb75ba41f0af85966021d1014f2d17e52f9" exitCode=0 Jul 24 13:15:32.478209 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:32.478127 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" event={"ID":"8d073917-4713-459e-9960-b177cacae466","Type":"ContainerDied","Data":"98b56430abea48e77c2cab88eb22bdb75ba41f0af85966021d1014f2d17e52f9"} Jul 24 13:15:33.484187 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.484152 2562 generic.go:356] "Generic (PLEG): container finished" podID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerID="9f6f4a2c8918b67fc4785470532ef6ab3b305069e69435c01627498a9dad2f1b" exitCode=0 Jul 24 13:15:33.484584 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.484219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" event={"ID":"9806a155-10e3-4b57-a897-d9443c7b68a4","Type":"ContainerDied","Data":"9f6f4a2c8918b67fc4785470532ef6ab3b305069e69435c01627498a9dad2f1b"} Jul 24 13:15:33.489198 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.489174 2562 generic.go:356] "Generic (PLEG): container finished" podID="8d073917-4713-459e-9960-b177cacae466" containerID="1f7f29a350d0754f3cb7e788ab999a369b4547c6f6dadf997e7d857fd2f803c6" exitCode=0 Jul 24 13:15:33.489334 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.489262 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" event={"ID":"8d073917-4713-459e-9960-b177cacae466","Type":"ContainerDied","Data":"1f7f29a350d0754f3cb7e788ab999a369b4547c6f6dadf997e7d857fd2f803c6"} Jul 24 13:15:33.490899 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.490879 2562 generic.go:356] "Generic (PLEG): container finished" podID="98f94866-c253-4b6b-8c0c-878220778359" containerID="b18221d57ce0631ce686127af2118b7e091516789a5138334dc45431e89964c2" exitCode=0 Jul 24 13:15:33.490986 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.490949 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" event={"ID":"98f94866-c253-4b6b-8c0c-878220778359","Type":"ContainerDied","Data":"b18221d57ce0631ce686127af2118b7e091516789a5138334dc45431e89964c2"} Jul 24 13:15:33.623187 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.623165 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:33.765421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.765355 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-bundle\") pod \"760249a8-7a07-496c-a5a0-c4f147034025\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " Jul 24 13:15:33.765421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.765387 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-util\") pod \"760249a8-7a07-496c-a5a0-c4f147034025\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " Jul 24 13:15:33.765421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.765419 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4lzb\" (UniqueName: \"kubernetes.io/projected/760249a8-7a07-496c-a5a0-c4f147034025-kube-api-access-b4lzb\") pod \"760249a8-7a07-496c-a5a0-c4f147034025\" (UID: \"760249a8-7a07-496c-a5a0-c4f147034025\") " Jul 24 13:15:33.765868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.765845 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-bundle" (OuterVolumeSpecName: "bundle") pod "760249a8-7a07-496c-a5a0-c4f147034025" (UID: "760249a8-7a07-496c-a5a0-c4f147034025"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:33.767619 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.767595 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760249a8-7a07-496c-a5a0-c4f147034025-kube-api-access-b4lzb" (OuterVolumeSpecName: "kube-api-access-b4lzb") pod "760249a8-7a07-496c-a5a0-c4f147034025" (UID: "760249a8-7a07-496c-a5a0-c4f147034025"). InnerVolumeSpecName "kube-api-access-b4lzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:15:33.772029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.771993 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-util" (OuterVolumeSpecName: "util") pod "760249a8-7a07-496c-a5a0-c4f147034025" (UID: "760249a8-7a07-496c-a5a0-c4f147034025"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:33.866176 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.866150 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:33.866176 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.866170 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760249a8-7a07-496c-a5a0-c4f147034025-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:33.866176 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:33.866178 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4lzb\" (UniqueName: \"kubernetes.io/projected/760249a8-7a07-496c-a5a0-c4f147034025-kube-api-access-b4lzb\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:34.497006 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.496975 2562 generic.go:356] "Generic (PLEG): container finished" podID="98f94866-c253-4b6b-8c0c-878220778359" containerID="43010023df3db52c3d7712d2ae2d0e9697eceeb32f1bd953d0bec5898c695814" exitCode=0 Jul 24 13:15:34.497409 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.497049 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" event={"ID":"98f94866-c253-4b6b-8c0c-878220778359","Type":"ContainerDied","Data":"43010023df3db52c3d7712d2ae2d0e9697eceeb32f1bd953d0bec5898c695814"} Jul 24 13:15:34.498688 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.498664 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" event={"ID":"760249a8-7a07-496c-a5a0-c4f147034025","Type":"ContainerDied","Data":"a9064638a14516b777494844a5f71f16c52fcef877cd83609a568361caad4ba9"} Jul 24 13:15:34.498806 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.498701 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9064638a14516b777494844a5f71f16c52fcef877cd83609a568361caad4ba9" Jul 24 13:15:34.499259 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.498986 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn" Jul 24 13:15:34.626897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.626876 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:34.662024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.661999 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:34.772242 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772171 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-bundle\") pod \"9806a155-10e3-4b57-a897-d9443c7b68a4\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " Jul 24 13:15:34.772242 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772213 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c62t8\" (UniqueName: \"kubernetes.io/projected/8d073917-4713-459e-9960-b177cacae466-kube-api-access-c62t8\") pod \"8d073917-4713-459e-9960-b177cacae466\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " Jul 24 13:15:34.772242 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772230 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfv64\" (UniqueName: \"kubernetes.io/projected/9806a155-10e3-4b57-a897-d9443c7b68a4-kube-api-access-vfv64\") pod \"9806a155-10e3-4b57-a897-d9443c7b68a4\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " Jul 24 13:15:34.772533 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772272 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-util\") pod \"9806a155-10e3-4b57-a897-d9443c7b68a4\" (UID: \"9806a155-10e3-4b57-a897-d9443c7b68a4\") " Jul 24 13:15:34.772533 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772306 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-util\") pod \"8d073917-4713-459e-9960-b177cacae466\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " Jul 24 13:15:34.772533 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772419 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-bundle\") pod \"8d073917-4713-459e-9960-b177cacae466\" (UID: \"8d073917-4713-459e-9960-b177cacae466\") " Jul 24 13:15:34.772974 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.772950 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-bundle" (OuterVolumeSpecName: "bundle") pod "9806a155-10e3-4b57-a897-d9443c7b68a4" (UID: "9806a155-10e3-4b57-a897-d9443c7b68a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:34.773054 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.773006 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-bundle" (OuterVolumeSpecName: "bundle") pod "8d073917-4713-459e-9960-b177cacae466" (UID: "8d073917-4713-459e-9960-b177cacae466"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:34.774378 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.774355 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d073917-4713-459e-9960-b177cacae466-kube-api-access-c62t8" (OuterVolumeSpecName: "kube-api-access-c62t8") pod "8d073917-4713-459e-9960-b177cacae466" (UID: "8d073917-4713-459e-9960-b177cacae466"). InnerVolumeSpecName "kube-api-access-c62t8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:15:34.774683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.774655 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9806a155-10e3-4b57-a897-d9443c7b68a4-kube-api-access-vfv64" (OuterVolumeSpecName: "kube-api-access-vfv64") pod "9806a155-10e3-4b57-a897-d9443c7b68a4" (UID: "9806a155-10e3-4b57-a897-d9443c7b68a4"). InnerVolumeSpecName "kube-api-access-vfv64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:15:34.778346 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.778314 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-util" (OuterVolumeSpecName: "util") pod "9806a155-10e3-4b57-a897-d9443c7b68a4" (UID: "9806a155-10e3-4b57-a897-d9443c7b68a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:34.779024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.779005 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-util" (OuterVolumeSpecName: "util") pod "8d073917-4713-459e-9960-b177cacae466" (UID: "8d073917-4713-459e-9960-b177cacae466"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:34.873275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.873251 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:34.873275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.873271 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:34.873275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.873280 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c62t8\" (UniqueName: \"kubernetes.io/projected/8d073917-4713-459e-9960-b177cacae466-kube-api-access-c62t8\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:34.873437 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.873289 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfv64\" (UniqueName: \"kubernetes.io/projected/9806a155-10e3-4b57-a897-d9443c7b68a4-kube-api-access-vfv64\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:34.873437 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.873297 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9806a155-10e3-4b57-a897-d9443c7b68a4-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:34.873437 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:34.873304 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8d073917-4713-459e-9960-b177cacae466-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:35.505347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.505320 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" Jul 24 13:15:35.505753 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.505347 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc" event={"ID":"9806a155-10e3-4b57-a897-d9443c7b68a4","Type":"ContainerDied","Data":"8c62fd8234a433877af69b98f0657e004c25bae9067a7e3ce61d1075acab3c1c"} Jul 24 13:15:35.505753 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.505391 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c62fd8234a433877af69b98f0657e004c25bae9067a7e3ce61d1075acab3c1c" Jul 24 13:15:35.507248 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.507232 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" Jul 24 13:15:35.507248 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.507224 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f" event={"ID":"8d073917-4713-459e-9960-b177cacae466","Type":"ContainerDied","Data":"4551d87d63e2ed6a504b369f690d1d057fe5c4ff4e9005c3b8ccc83296fc98d7"} Jul 24 13:15:35.507392 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.507274 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4551d87d63e2ed6a504b369f690d1d057fe5c4ff4e9005c3b8ccc83296fc98d7" Jul 24 13:15:35.634139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.634117 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:35.782053 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.781973 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vlbz\" (UniqueName: \"kubernetes.io/projected/98f94866-c253-4b6b-8c0c-878220778359-kube-api-access-7vlbz\") pod \"98f94866-c253-4b6b-8c0c-878220778359\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " Jul 24 13:15:35.782053 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.782038 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-bundle\") pod \"98f94866-c253-4b6b-8c0c-878220778359\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " Jul 24 13:15:35.782276 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.782067 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-util\") pod \"98f94866-c253-4b6b-8c0c-878220778359\" (UID: \"98f94866-c253-4b6b-8c0c-878220778359\") " Jul 24 13:15:35.782454 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.782426 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-bundle" (OuterVolumeSpecName: "bundle") pod "98f94866-c253-4b6b-8c0c-878220778359" (UID: "98f94866-c253-4b6b-8c0c-878220778359"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:35.784083 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.784058 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f94866-c253-4b6b-8c0c-878220778359-kube-api-access-7vlbz" (OuterVolumeSpecName: "kube-api-access-7vlbz") pod "98f94866-c253-4b6b-8c0c-878220778359" (UID: "98f94866-c253-4b6b-8c0c-878220778359"). InnerVolumeSpecName "kube-api-access-7vlbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:15:35.787606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.787582 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-util" (OuterVolumeSpecName: "util") pod "98f94866-c253-4b6b-8c0c-878220778359" (UID: "98f94866-c253-4b6b-8c0c-878220778359"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:15:35.883017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.882974 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vlbz\" (UniqueName: \"kubernetes.io/projected/98f94866-c253-4b6b-8c0c-878220778359-kube-api-access-7vlbz\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:35.883017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.883012 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:35.883017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:35.883022 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98f94866-c253-4b6b-8c0c-878220778359-util\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:15:36.517532 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:36.517492 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" event={"ID":"98f94866-c253-4b6b-8c0c-878220778359","Type":"ContainerDied","Data":"d0b47512392d6591ab047abc2fcb95ad472743964d9cb639e7ada435b301f95f"} Jul 24 13:15:36.517938 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:36.517515 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p" Jul 24 13:15:36.517938 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:15:36.517592 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0b47512392d6591ab047abc2fcb95ad472743964d9cb639e7ada435b301f95f" Jul 24 13:16:04.516585 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:04.516536 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-564899d58c-jssfd"] Jul 24 13:16:29.535561 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.535494 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-564899d58c-jssfd" podUID="221f5a62-7982-4798-b116-29dc599e5dca" containerName="console" containerID="cri-o://1c914d6251fa2863cf8b02611d9b8f67568f7b25952555d2924213978da81380" gracePeriod=15 Jul 24 13:16:29.760970 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.760932 2562 generic.go:356] "Generic (PLEG): container finished" podID="221f5a62-7982-4798-b116-29dc599e5dca" containerID="1c914d6251fa2863cf8b02611d9b8f67568f7b25952555d2924213978da81380" exitCode=0 Jul 24 13:16:29.761125 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.761015 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564899d58c-jssfd" event={"ID":"221f5a62-7982-4798-b116-29dc599e5dca","Type":"ContainerDied","Data":"1c914d6251fa2863cf8b02611d9b8f67568f7b25952555d2924213978da81380"} Jul 24 13:16:29.782573 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.782537 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:16:29.832455 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832392 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-serving-cert\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832455 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832439 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-service-ca\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832460 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-oauth-config\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832477 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-trusted-ca-bundle\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832511 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-console-config\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832580 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fp5gd\" (UniqueName: \"kubernetes.io/projected/221f5a62-7982-4798-b116-29dc599e5dca-kube-api-access-fp5gd\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832607 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-oauth-serving-cert\") pod \"221f5a62-7982-4798-b116-29dc599e5dca\" (UID: \"221f5a62-7982-4798-b116-29dc599e5dca\") " Jul 24 13:16:29.832971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.832916 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-service-ca" (OuterVolumeSpecName: "service-ca") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:16:29.833060 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.833033 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:16:29.833125 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.833060 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-console-config" (OuterVolumeSpecName: "console-config") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:16:29.833179 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.833138 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:16:29.834643 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.834613 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:16:29.834745 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.834683 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:16:29.834745 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.834670 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/221f5a62-7982-4798-b116-29dc599e5dca-kube-api-access-fp5gd" (OuterVolumeSpecName: "kube-api-access-fp5gd") pod "221f5a62-7982-4798-b116-29dc599e5dca" (UID: "221f5a62-7982-4798-b116-29dc599e5dca"). InnerVolumeSpecName "kube-api-access-fp5gd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:16:29.933344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933320 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:29.933344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933340 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-oauth-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:29.933495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933350 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-trusted-ca-bundle\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:29.933495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933359 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-console-config\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:29.933495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933367 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fp5gd\" (UniqueName: \"kubernetes.io/projected/221f5a62-7982-4798-b116-29dc599e5dca-kube-api-access-fp5gd\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:29.933495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933376 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/221f5a62-7982-4798-b116-29dc599e5dca-oauth-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:29.933495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:29.933386 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/221f5a62-7982-4798-b116-29dc599e5dca-console-serving-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:16:30.766536 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:30.766497 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-564899d58c-jssfd" Jul 24 13:16:30.767017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:30.766531 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-564899d58c-jssfd" event={"ID":"221f5a62-7982-4798-b116-29dc599e5dca","Type":"ContainerDied","Data":"074bf18047eacf37f2b6490304e60744bcc2d9d028f6c668bb63ad04910a9cbd"} Jul 24 13:16:30.767017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:30.766615 2562 scope.go:117] "RemoveContainer" containerID="1c914d6251fa2863cf8b02611d9b8f67568f7b25952555d2924213978da81380" Jul 24 13:16:30.783808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:30.783788 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-564899d58c-jssfd"] Jul 24 13:16:30.787746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:30.787726 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-564899d58c-jssfd"] Jul 24 13:16:31.895790 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:31.895756 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="221f5a62-7982-4798-b116-29dc599e5dca" path="/var/lib/kubelet/pods/221f5a62-7982-4798-b116-29dc599e5dca/volumes" Jul 24 13:16:37.020905 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.020876 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr"] Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021201 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="pull" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021213 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="pull" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021223 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="pull" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021228 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="pull" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021234 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="pull" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021239 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="pull" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021244 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="extract" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021250 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="extract" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021258 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="221f5a62-7982-4798-b116-29dc599e5dca" containerName="console" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021263 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="221f5a62-7982-4798-b116-29dc599e5dca" containerName="console" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021270 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="util" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021275 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="util" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021282 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="extract" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021287 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="extract" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021296 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="util" Jul 24 13:16:37.021293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021301 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="util" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021308 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021313 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021320 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021325 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021331 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="pull" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021336 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="pull" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021341 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="util" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021346 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="util" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021352 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="util" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021356 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="util" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021400 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9806a155-10e3-4b57-a897-d9443c7b68a4" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021407 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d073917-4713-459e-9960-b177cacae466" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021413 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="760249a8-7a07-496c-a5a0-c4f147034025" containerName="extract" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021419 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="221f5a62-7982-4798-b116-29dc599e5dca" containerName="console" Jul 24 13:16:37.021829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.021423 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f94866-c253-4b6b-8c0c-878220778359" containerName="extract" Jul 24 13:16:37.025789 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.025774 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.028280 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.028263 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xbbpn\"" Jul 24 13:16:37.042009 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.041984 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr"] Jul 24 13:16:37.082470 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082445 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082592 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082592 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082592 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082519 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082592 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082592 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvd2\" (UniqueName: \"kubernetes.io/projected/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-kube-api-access-fmvd2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082845 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082845 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.082845 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.082749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183412 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183412 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183414 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183667 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183434 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183667 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183667 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183590 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvd2\" (UniqueName: \"kubernetes.io/projected/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-kube-api-access-fmvd2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183667 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.183864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183786 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.184021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183866 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.184021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.183930 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.184137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.184058 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.184137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.184078 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.184241 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.184189 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.185976 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.185950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.186089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.186008 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.197207 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.197183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvd2\" (UniqueName: \"kubernetes.io/projected/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-kube-api-access-fmvd2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.197299 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.197240 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7de505d-0a1c-4d53-9bc6-79eea2973b8e-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-kcjwr\" (UID: \"b7de505d-0a1c-4d53-9bc6-79eea2973b8e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.336256 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.336174 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:37.476985 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.476959 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr"] Jul 24 13:16:37.479080 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:16:37.479043 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7de505d_0a1c_4d53_9bc6_79eea2973b8e.slice/crio-024162f1b59d3630488c6455cf6d7a69a8113381c490f4320c74bcdd2cb6028b WatchSource:0}: Error finding container 024162f1b59d3630488c6455cf6d7a69a8113381c490f4320c74bcdd2cb6028b: Status 404 returned error can't find the container with id 024162f1b59d3630488c6455cf6d7a69a8113381c490f4320c74bcdd2cb6028b Jul 24 13:16:37.482000 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.481956 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:16:37.482145 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.482111 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:16:37.482225 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.482194 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:16:37.799261 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.799227 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" event={"ID":"b7de505d-0a1c-4d53-9bc6-79eea2973b8e","Type":"ContainerStarted","Data":"8060f2fd220c4c1626ffc093ce83cb84466700e029ebfae00895f1553acd66b8"} Jul 24 13:16:37.799431 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.799307 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" event={"ID":"b7de505d-0a1c-4d53-9bc6-79eea2973b8e","Type":"ContainerStarted","Data":"024162f1b59d3630488c6455cf6d7a69a8113381c490f4320c74bcdd2cb6028b"} Jul 24 13:16:37.824148 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:37.824097 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" podStartSLOduration=0.824080453 podStartE2EDuration="824.080453ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:16:37.822307128 +0000 UTC m=+424.421552338" watchObservedRunningTime="2026-07-24 13:16:37.824080453 +0000 UTC m=+424.423325663" Jul 24 13:16:38.336519 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:38.336484 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:38.341431 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:38.341405 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:38.804408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:38.804381 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:38.805385 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:38.805365 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:16:40.036243 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.036209 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-s58dm"] Jul 24 13:16:40.039579 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.039563 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:16:40.041717 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.041693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jfbtl\"" Jul 24 13:16:40.041828 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.041694 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 13:16:40.041828 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.041696 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 13:16:40.045954 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.045925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s58dm"] Jul 24 13:16:40.107518 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.107494 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbw6\" (UniqueName: \"kubernetes.io/projected/01a2485d-6367-4574-8432-95e690949ccf-kube-api-access-7kbw6\") pod \"authorino-75d556bd77-s58dm\" (UID: \"01a2485d-6367-4574-8432-95e690949ccf\") " pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:16:40.208241 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.208212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbw6\" (UniqueName: \"kubernetes.io/projected/01a2485d-6367-4574-8432-95e690949ccf-kube-api-access-7kbw6\") pod \"authorino-75d556bd77-s58dm\" (UID: \"01a2485d-6367-4574-8432-95e690949ccf\") " pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:16:40.216804 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.216777 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbw6\" (UniqueName: \"kubernetes.io/projected/01a2485d-6367-4574-8432-95e690949ccf-kube-api-access-7kbw6\") pod \"authorino-75d556bd77-s58dm\" (UID: \"01a2485d-6367-4574-8432-95e690949ccf\") " pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:16:40.350256 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.350182 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:16:40.491253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.491223 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s58dm"] Jul 24 13:16:40.494196 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:16:40.494168 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01a2485d_6367_4574_8432_95e690949ccf.slice/crio-80952b75fae64483c0e258626985508e6c65068d7dc455025ea0f0b2c778b053 WatchSource:0}: Error finding container 80952b75fae64483c0e258626985508e6c65068d7dc455025ea0f0b2c778b053: Status 404 returned error can't find the container with id 80952b75fae64483c0e258626985508e6c65068d7dc455025ea0f0b2c778b053 Jul 24 13:16:40.815429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:40.815396 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s58dm" event={"ID":"01a2485d-6367-4574-8432-95e690949ccf","Type":"ContainerStarted","Data":"80952b75fae64483c0e258626985508e6c65068d7dc455025ea0f0b2c778b053"} Jul 24 13:16:43.834606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:43.834571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s58dm" event={"ID":"01a2485d-6367-4574-8432-95e690949ccf","Type":"ContainerStarted","Data":"141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4"} Jul 24 13:16:43.853450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:16:43.853359 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-s58dm" podStartSLOduration=0.755864459 podStartE2EDuration="3.853343799s" totalImagesPullingTime="3.09747934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:16:43.85074541 +0000 UTC m=+430.449990622" watchObservedRunningTime="2026-07-24 13:16:43.853343799 +0000 UTC m=+430.452589044" Jul 24 13:17:09.368205 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.368124 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s58dm"] Jul 24 13:17:09.368722 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.368323 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-s58dm" podUID="01a2485d-6367-4574-8432-95e690949ccf" containerName="authorino" containerID="cri-o://141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4" gracePeriod=30 Jul 24 13:17:09.615115 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.615090 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:17:09.751825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.751794 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kbw6\" (UniqueName: \"kubernetes.io/projected/01a2485d-6367-4574-8432-95e690949ccf-kube-api-access-7kbw6\") pod \"01a2485d-6367-4574-8432-95e690949ccf\" (UID: \"01a2485d-6367-4574-8432-95e690949ccf\") " Jul 24 13:17:09.753969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.753941 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a2485d-6367-4574-8432-95e690949ccf-kube-api-access-7kbw6" (OuterVolumeSpecName: "kube-api-access-7kbw6") pod "01a2485d-6367-4574-8432-95e690949ccf" (UID: "01a2485d-6367-4574-8432-95e690949ccf"). InnerVolumeSpecName "kube-api-access-7kbw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:17:09.853262 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.853234 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kbw6\" (UniqueName: \"kubernetes.io/projected/01a2485d-6367-4574-8432-95e690949ccf-kube-api-access-7kbw6\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:17:09.958090 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.958060 2562 generic.go:356] "Generic (PLEG): container finished" podID="01a2485d-6367-4574-8432-95e690949ccf" containerID="141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4" exitCode=0 Jul 24 13:17:09.958209 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.958097 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-s58dm" Jul 24 13:17:09.958209 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.958106 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s58dm" event={"ID":"01a2485d-6367-4574-8432-95e690949ccf","Type":"ContainerDied","Data":"141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4"} Jul 24 13:17:09.958209 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.958132 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-s58dm" event={"ID":"01a2485d-6367-4574-8432-95e690949ccf","Type":"ContainerDied","Data":"80952b75fae64483c0e258626985508e6c65068d7dc455025ea0f0b2c778b053"} Jul 24 13:17:09.958209 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.958148 2562 scope.go:117] "RemoveContainer" containerID="141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4" Jul 24 13:17:09.966392 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.966374 2562 scope.go:117] "RemoveContainer" containerID="141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4" Jul 24 13:17:09.966686 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:17:09.966668 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4\": container with ID starting with 141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4 not found: ID does not exist" containerID="141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4" Jul 24 13:17:09.966738 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.966694 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4"} err="failed to get container status \"141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4\": rpc error: code = NotFound desc = could not find container \"141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4\": container with ID starting with 141539c24148eadb29f9330f028086ca827c3110fbd0696157682db5abb7f2a4 not found: ID does not exist" Jul 24 13:17:09.972211 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.972192 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s58dm"] Jul 24 13:17:09.975696 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:09.975678 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-s58dm"] Jul 24 13:17:11.895679 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:11.895643 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a2485d-6367-4574-8432-95e690949ccf" path="/var/lib/kubelet/pods/01a2485d-6367-4574-8432-95e690949ccf/volumes" Jul 24 13:17:15.264291 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.264255 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-78d55487c8-47mxs"] Jul 24 13:17:15.264781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.264575 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a2485d-6367-4574-8432-95e690949ccf" containerName="authorino" Jul 24 13:17:15.264781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.264588 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a2485d-6367-4574-8432-95e690949ccf" containerName="authorino" Jul 24 13:17:15.264781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.264686 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a2485d-6367-4574-8432-95e690949ccf" containerName="authorino" Jul 24 13:17:15.267913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.267893 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.270295 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.270276 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:17:15.270433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.270413 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nbq5n\"" Jul 24 13:17:15.271170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.271151 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:17:15.281318 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.281288 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78d55487c8-47mxs"] Jul 24 13:17:15.398694 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.398659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c3fd89-db63-4cda-9830-eb7a8523c6a4-webhook-cert\") pod \"maas-controller-78d55487c8-47mxs\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.398867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.398769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcdbx\" (UniqueName: \"kubernetes.io/projected/79c3fd89-db63-4cda-9830-eb7a8523c6a4-kube-api-access-xcdbx\") pod \"maas-controller-78d55487c8-47mxs\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.427226 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.427193 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf5655576-n2xbm"] Jul 24 13:17:15.430309 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.430285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.438852 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.438822 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5655576-n2xbm"] Jul 24 13:17:15.499726 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.499693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcdbx\" (UniqueName: \"kubernetes.io/projected/79c3fd89-db63-4cda-9830-eb7a8523c6a4-kube-api-access-xcdbx\") pod \"maas-controller-78d55487c8-47mxs\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.499892 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.499747 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c3fd89-db63-4cda-9830-eb7a8523c6a4-webhook-cert\") pod \"maas-controller-78d55487c8-47mxs\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.502100 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.502079 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c3fd89-db63-4cda-9830-eb7a8523c6a4-webhook-cert\") pod \"maas-controller-78d55487c8-47mxs\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.506516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.506495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcdbx\" (UniqueName: \"kubernetes.io/projected/79c3fd89-db63-4cda-9830-eb7a8523c6a4-kube-api-access-xcdbx\") pod \"maas-controller-78d55487c8-47mxs\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.578616 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.578529 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:15.600709 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.600684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68qf4\" (UniqueName: \"kubernetes.io/projected/a50c8166-01a6-4d61-99c6-f37d82844faa-kube-api-access-68qf4\") pod \"maas-controller-6cf5655576-n2xbm\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.600804 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.600782 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a50c8166-01a6-4d61-99c6-f37d82844faa-webhook-cert\") pod \"maas-controller-6cf5655576-n2xbm\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.701199 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.701139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68qf4\" (UniqueName: \"kubernetes.io/projected/a50c8166-01a6-4d61-99c6-f37d82844faa-kube-api-access-68qf4\") pod \"maas-controller-6cf5655576-n2xbm\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.701359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.701245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a50c8166-01a6-4d61-99c6-f37d82844faa-webhook-cert\") pod \"maas-controller-6cf5655576-n2xbm\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.703972 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.703949 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a50c8166-01a6-4d61-99c6-f37d82844faa-webhook-cert\") pod \"maas-controller-6cf5655576-n2xbm\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.705645 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.705623 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-78d55487c8-47mxs"] Jul 24 13:17:15.707396 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:17:15.707363 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c3fd89_db63_4cda_9830_eb7a8523c6a4.slice/crio-7c58fe39710a053f9678343142e79f0fcb0cb9df4dc3c847bd03d2a61eec7a34 WatchSource:0}: Error finding container 7c58fe39710a053f9678343142e79f0fcb0cb9df4dc3c847bd03d2a61eec7a34: Status 404 returned error can't find the container with id 7c58fe39710a053f9678343142e79f0fcb0cb9df4dc3c847bd03d2a61eec7a34 Jul 24 13:17:15.709202 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.709183 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68qf4\" (UniqueName: \"kubernetes.io/projected/a50c8166-01a6-4d61-99c6-f37d82844faa-kube-api-access-68qf4\") pod \"maas-controller-6cf5655576-n2xbm\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.741269 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.741242 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:15.868472 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.868447 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5655576-n2xbm"] Jul 24 13:17:15.870237 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:17:15.870211 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50c8166_01a6_4d61_99c6_f37d82844faa.slice/crio-3fd751796f87334eae0b60e549768e3aef357b4d2a7879a244f3a35357232309 WatchSource:0}: Error finding container 3fd751796f87334eae0b60e549768e3aef357b4d2a7879a244f3a35357232309: Status 404 returned error can't find the container with id 3fd751796f87334eae0b60e549768e3aef357b4d2a7879a244f3a35357232309 Jul 24 13:17:15.987606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.987562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-n2xbm" event={"ID":"a50c8166-01a6-4d61-99c6-f37d82844faa","Type":"ContainerStarted","Data":"3fd751796f87334eae0b60e549768e3aef357b4d2a7879a244f3a35357232309"} Jul 24 13:17:15.988672 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:15.988651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78d55487c8-47mxs" event={"ID":"79c3fd89-db63-4cda-9830-eb7a8523c6a4","Type":"ContainerStarted","Data":"7c58fe39710a053f9678343142e79f0fcb0cb9df4dc3c847bd03d2a61eec7a34"} Jul 24 13:17:19.010614 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:19.010577 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-n2xbm" event={"ID":"a50c8166-01a6-4d61-99c6-f37d82844faa","Type":"ContainerStarted","Data":"86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43"} Jul 24 13:17:19.011073 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:19.010676 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:19.012138 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:19.012108 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78d55487c8-47mxs" event={"ID":"79c3fd89-db63-4cda-9830-eb7a8523c6a4","Type":"ContainerStarted","Data":"45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7"} Jul 24 13:17:19.012218 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:19.012190 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:19.029903 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:19.029852 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf5655576-n2xbm" podStartSLOduration=1.5756041280000002 podStartE2EDuration="4.029840498s" totalImagesPullingTime="2.45423637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:17:19.027492435 +0000 UTC m=+465.626737648" watchObservedRunningTime="2026-07-24 13:17:19.029840498 +0000 UTC m=+465.629085708" Jul 24 13:17:19.045056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:19.045018 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-78d55487c8-47mxs" podStartSLOduration=1.436025271 podStartE2EDuration="4.045005933s" totalImagesPullingTime="2.608980662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:17:19.044111921 +0000 UTC m=+465.643357143" watchObservedRunningTime="2026-07-24 13:17:19.045005933 +0000 UTC m=+465.644251187" Jul 24 13:17:30.018372 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.018343 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:17:30.018755 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.018495 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:30.088490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.088451 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78d55487c8-47mxs"] Jul 24 13:17:30.088713 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.088686 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-78d55487c8-47mxs" podUID="79c3fd89-db63-4cda-9830-eb7a8523c6a4" containerName="manager" containerID="cri-o://45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7" gracePeriod=30 Jul 24 13:17:30.328926 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.328904 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:30.421206 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.421179 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcdbx\" (UniqueName: \"kubernetes.io/projected/79c3fd89-db63-4cda-9830-eb7a8523c6a4-kube-api-access-xcdbx\") pod \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " Jul 24 13:17:30.421345 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.421218 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c3fd89-db63-4cda-9830-eb7a8523c6a4-webhook-cert\") pod \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\" (UID: \"79c3fd89-db63-4cda-9830-eb7a8523c6a4\") " Jul 24 13:17:30.423372 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.423340 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79c3fd89-db63-4cda-9830-eb7a8523c6a4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "79c3fd89-db63-4cda-9830-eb7a8523c6a4" (UID: "79c3fd89-db63-4cda-9830-eb7a8523c6a4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:17:30.423372 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.423342 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c3fd89-db63-4cda-9830-eb7a8523c6a4-kube-api-access-xcdbx" (OuterVolumeSpecName: "kube-api-access-xcdbx") pod "79c3fd89-db63-4cda-9830-eb7a8523c6a4" (UID: "79c3fd89-db63-4cda-9830-eb7a8523c6a4"). InnerVolumeSpecName "kube-api-access-xcdbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:17:30.521977 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.521948 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcdbx\" (UniqueName: \"kubernetes.io/projected/79c3fd89-db63-4cda-9830-eb7a8523c6a4-kube-api-access-xcdbx\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:17:30.521977 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:30.521972 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79c3fd89-db63-4cda-9830-eb7a8523c6a4-webhook-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:17:31.069012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.068975 2562 generic.go:356] "Generic (PLEG): container finished" podID="79c3fd89-db63-4cda-9830-eb7a8523c6a4" containerID="45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7" exitCode=0 Jul 24 13:17:31.069461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.069034 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-78d55487c8-47mxs" Jul 24 13:17:31.069461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.069048 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78d55487c8-47mxs" event={"ID":"79c3fd89-db63-4cda-9830-eb7a8523c6a4","Type":"ContainerDied","Data":"45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7"} Jul 24 13:17:31.069461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.069100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-78d55487c8-47mxs" event={"ID":"79c3fd89-db63-4cda-9830-eb7a8523c6a4","Type":"ContainerDied","Data":"7c58fe39710a053f9678343142e79f0fcb0cb9df4dc3c847bd03d2a61eec7a34"} Jul 24 13:17:31.069461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.069122 2562 scope.go:117] "RemoveContainer" containerID="45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7" Jul 24 13:17:31.078935 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.078911 2562 scope.go:117] "RemoveContainer" containerID="45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7" Jul 24 13:17:31.079206 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:17:31.079186 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7\": container with ID starting with 45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7 not found: ID does not exist" containerID="45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7" Jul 24 13:17:31.079292 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.079217 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7"} err="failed to get container status \"45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7\": rpc error: code = NotFound desc = could not find container \"45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7\": container with ID starting with 45e3e4de3b7013c3fe924342c9c66464b4679a4a7a8fc4d07ff5bbaac1d6b0c7 not found: ID does not exist" Jul 24 13:17:31.090355 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.090335 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-78d55487c8-47mxs"] Jul 24 13:17:31.096446 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.096423 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-78d55487c8-47mxs"] Jul 24 13:17:31.895839 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:31.895807 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c3fd89-db63-4cda-9830-eb7a8523c6a4" path="/var/lib/kubelet/pods/79c3fd89-db63-4cda-9830-eb7a8523c6a4/volumes" Jul 24 13:17:51.770807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.770779 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54b587c679-mcfgw"] Jul 24 13:17:51.771253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.771099 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c3fd89-db63-4cda-9830-eb7a8523c6a4" containerName="manager" Jul 24 13:17:51.771253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.771110 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c3fd89-db63-4cda-9830-eb7a8523c6a4" containerName="manager" Jul 24 13:17:51.771253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.771162 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c3fd89-db63-4cda-9830-eb7a8523c6a4" containerName="manager" Jul 24 13:17:51.779774 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.779751 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.782143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.782115 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 24 13:17:51.782257 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.782152 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 24 13:17:51.782586 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.782566 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 13:17:51.784416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.784397 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b587c679-mcfgw"] Jul 24 13:17:51.886785 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.886751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrr5\" (UniqueName: \"kubernetes.io/projected/b551d8fb-725b-4988-a4c1-5b4df7cfa981-kube-api-access-xvrr5\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.886785 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.886788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551d8fb-725b-4988-a4c1-5b4df7cfa981-maas-api-tls\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.886953 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.886812 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551d8fb-725b-4988-a4c1-5b4df7cfa981-openshift-service-ca\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.988116 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.988086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrr5\" (UniqueName: \"kubernetes.io/projected/b551d8fb-725b-4988-a4c1-5b4df7cfa981-kube-api-access-xvrr5\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.988259 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.988124 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551d8fb-725b-4988-a4c1-5b4df7cfa981-maas-api-tls\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.988259 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.988158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551d8fb-725b-4988-a4c1-5b4df7cfa981-openshift-service-ca\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.988797 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.988780 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551d8fb-725b-4988-a4c1-5b4df7cfa981-openshift-service-ca\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.990470 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.990450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551d8fb-725b-4988-a4c1-5b4df7cfa981-maas-api-tls\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:51.996237 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:51.996215 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrr5\" (UniqueName: \"kubernetes.io/projected/b551d8fb-725b-4988-a4c1-5b4df7cfa981-kube-api-access-xvrr5\") pod \"maas-api-54b587c679-mcfgw\" (UID: \"b551d8fb-725b-4988-a4c1-5b4df7cfa981\") " pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:52.091393 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.091313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:52.236285 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.236256 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54b587c679-mcfgw"] Jul 24 13:17:52.240226 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:17:52.240192 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb551d8fb_725b_4988_a4c1_5b4df7cfa981.slice/crio-ec2a7405e01e0d30c27ea141953e52c924f2b9853493d6fb6f95ce95b9fef0b2 WatchSource:0}: Error finding container ec2a7405e01e0d30c27ea141953e52c924f2b9853493d6fb6f95ce95b9fef0b2: Status 404 returned error can't find the container with id ec2a7405e01e0d30c27ea141953e52c924f2b9853493d6fb6f95ce95b9fef0b2 Jul 24 13:17:52.329393 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.329363 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz"] Jul 24 13:17:52.334360 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.334342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.337279 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.337260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 24 13:17:52.349732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.349663 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz"] Jul 24 13:17:52.365261 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.365244 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-v4mkx"] Jul 24 13:17:52.368862 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.368843 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.371227 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.371211 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-qdvzk\"" Jul 24 13:17:52.385471 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.385450 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-v4mkx"] Jul 24 13:17:52.392691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.392672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24789\" (UniqueName: \"kubernetes.io/projected/3edbf7e3-9350-4ede-a85e-8fccbf657c1e-kube-api-access-24789\") pod \"payload-pre-processing-687dbb6cf6-4tzvz\" (UID: \"3edbf7e3-9350-4ede-a85e-8fccbf657c1e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.392774 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.392714 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3edbf7e3-9350-4ede-a85e-8fccbf657c1e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4tzvz\" (UID: \"3edbf7e3-9350-4ede-a85e-8fccbf657c1e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.494157 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.494129 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3edbf7e3-9350-4ede-a85e-8fccbf657c1e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4tzvz\" (UID: \"3edbf7e3-9350-4ede-a85e-8fccbf657c1e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.494336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.494185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49addf5c-c5ab-446f-af41-7447ff877af5-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-v4mkx\" (UID: \"49addf5c-c5ab-446f-af41-7447ff877af5\") " pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.494336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.494222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24789\" (UniqueName: \"kubernetes.io/projected/3edbf7e3-9350-4ede-a85e-8fccbf657c1e-kube-api-access-24789\") pod \"payload-pre-processing-687dbb6cf6-4tzvz\" (UID: \"3edbf7e3-9350-4ede-a85e-8fccbf657c1e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.494336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.494244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpzpc\" (UniqueName: \"kubernetes.io/projected/49addf5c-c5ab-446f-af41-7447ff877af5-kube-api-access-gpzpc\") pod \"payload-processing-5cc6cd444b-v4mkx\" (UID: \"49addf5c-c5ab-446f-af41-7447ff877af5\") " pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.494868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.494845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3edbf7e3-9350-4ede-a85e-8fccbf657c1e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-4tzvz\" (UID: \"3edbf7e3-9350-4ede-a85e-8fccbf657c1e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.515107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.515084 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24789\" (UniqueName: \"kubernetes.io/projected/3edbf7e3-9350-4ede-a85e-8fccbf657c1e-kube-api-access-24789\") pod \"payload-pre-processing-687dbb6cf6-4tzvz\" (UID: \"3edbf7e3-9350-4ede-a85e-8fccbf657c1e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.595562 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.595516 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49addf5c-c5ab-446f-af41-7447ff877af5-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-v4mkx\" (UID: \"49addf5c-c5ab-446f-af41-7447ff877af5\") " pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.595703 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.595580 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpzpc\" (UniqueName: \"kubernetes.io/projected/49addf5c-c5ab-446f-af41-7447ff877af5-kube-api-access-gpzpc\") pod \"payload-processing-5cc6cd444b-v4mkx\" (UID: \"49addf5c-c5ab-446f-af41-7447ff877af5\") " pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.596214 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.596196 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49addf5c-c5ab-446f-af41-7447ff877af5-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-v4mkx\" (UID: \"49addf5c-c5ab-446f-af41-7447ff877af5\") " pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.603750 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.603689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpzpc\" (UniqueName: \"kubernetes.io/projected/49addf5c-c5ab-446f-af41-7447ff877af5-kube-api-access-gpzpc\") pod \"payload-processing-5cc6cd444b-v4mkx\" (UID: \"49addf5c-c5ab-446f-af41-7447ff877af5\") " pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.644661 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.644626 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:52.678467 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.678413 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:52.783146 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.783093 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz"] Jul 24 13:17:52.786190 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:17:52.786159 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3edbf7e3_9350_4ede_a85e_8fccbf657c1e.slice/crio-471a38dc355a1af7304ef3aa48685bbb059ecb684f116523fd24a81a71ebf4c1 WatchSource:0}: Error finding container 471a38dc355a1af7304ef3aa48685bbb059ecb684f116523fd24a81a71ebf4c1: Status 404 returned error can't find the container with id 471a38dc355a1af7304ef3aa48685bbb059ecb684f116523fd24a81a71ebf4c1 Jul 24 13:17:52.816782 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:52.816762 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-v4mkx"] Jul 24 13:17:52.818700 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:17:52.818678 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49addf5c_c5ab_446f_af41_7447ff877af5.slice/crio-de84aa239cb4eff95076425bfd2df032cb1b75eb04ff075f1590c2aa1459c37b WatchSource:0}: Error finding container de84aa239cb4eff95076425bfd2df032cb1b75eb04ff075f1590c2aa1459c37b: Status 404 returned error can't find the container with id de84aa239cb4eff95076425bfd2df032cb1b75eb04ff075f1590c2aa1459c37b Jul 24 13:17:53.175712 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:53.175671 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" event={"ID":"49addf5c-c5ab-446f-af41-7447ff877af5","Type":"ContainerStarted","Data":"de84aa239cb4eff95076425bfd2df032cb1b75eb04ff075f1590c2aa1459c37b"} Jul 24 13:17:53.176885 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:53.176855 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" event={"ID":"3edbf7e3-9350-4ede-a85e-8fccbf657c1e","Type":"ContainerStarted","Data":"471a38dc355a1af7304ef3aa48685bbb059ecb684f116523fd24a81a71ebf4c1"} Jul 24 13:17:53.178094 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:53.178064 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" event={"ID":"b551d8fb-725b-4988-a4c1-5b4df7cfa981","Type":"ContainerStarted","Data":"ec2a7405e01e0d30c27ea141953e52c924f2b9853493d6fb6f95ce95b9fef0b2"} Jul 24 13:17:55.199875 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:55.199781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" event={"ID":"b551d8fb-725b-4988-a4c1-5b4df7cfa981","Type":"ContainerStarted","Data":"2de38f2daa0950eacf77e43590d90d8ce2e4a5e8178cbc1c81879b83546db454"} Jul 24 13:17:55.200965 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:55.200903 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:17:56.208675 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.208636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" event={"ID":"49addf5c-c5ab-446f-af41-7447ff877af5","Type":"ContainerStarted","Data":"f8229d176add910d6b1a951a913dda31ada466e6125d2a1f7540a6cae19165c8"} Jul 24 13:17:56.210193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.210170 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:17:56.212778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.212753 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" event={"ID":"3edbf7e3-9350-4ede-a85e-8fccbf657c1e","Type":"ContainerStarted","Data":"1852fb15d6698e5d70f81d028debe75abd33f8fa1df4eeedba472e6cd0f4de6b"} Jul 24 13:17:56.213843 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.213829 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:17:56.225607 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.225563 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" podStartSLOduration=2.966830708 podStartE2EDuration="5.225536375s" totalImagesPullingTime="2.258705667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:17:55.223412133 +0000 UTC m=+501.822657345" watchObservedRunningTime="2026-07-24 13:17:56.225536375 +0000 UTC m=+502.824781564" Jul 24 13:17:56.226315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.226288 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" podStartSLOduration=1.862643152 podStartE2EDuration="4.226281427s" totalImagesPullingTime="2.363638275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:17:56.224038868 +0000 UTC m=+502.823284080" watchObservedRunningTime="2026-07-24 13:17:56.226281427 +0000 UTC m=+502.825526639" Jul 24 13:17:56.247246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:17:56.247209 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" podStartSLOduration=1.857174535 podStartE2EDuration="4.247197153s" totalImagesPullingTime="2.390022618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:17:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:17:56.24470062 +0000 UTC m=+502.843945835" watchObservedRunningTime="2026-07-24 13:17:56.247197153 +0000 UTC m=+502.846442363" Jul 24 13:18:00.235918 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:00.235885 2562 generic.go:356] "Generic (PLEG): container finished" podID="b7de505d-0a1c-4d53-9bc6-79eea2973b8e" containerID="8060f2fd220c4c1626ffc093ce83cb84466700e029ebfae00895f1553acd66b8" exitCode=0 Jul 24 13:18:00.236360 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:00.235947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" event={"ID":"b7de505d-0a1c-4d53-9bc6-79eea2973b8e","Type":"ContainerDied","Data":"8060f2fd220c4c1626ffc093ce83cb84466700e029ebfae00895f1553acd66b8"} Jul 24 13:18:00.236360 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:00.236295 2562 scope.go:117] "RemoveContainer" containerID="8060f2fd220c4c1626ffc093ce83cb84466700e029ebfae00895f1553acd66b8" Jul 24 13:18:00.237206 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:00.237178 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:18:00.237254 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:00.237243 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:18:00.237288 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:00.237275 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:18:01.219330 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:01.219302 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54b587c679-mcfgw" Jul 24 13:18:01.247505 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:01.247468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" event={"ID":"b7de505d-0a1c-4d53-9bc6-79eea2973b8e","Type":"ContainerStarted","Data":"d740a2e378d4d378f3f426bf56f4042fa734211809f6b43e10faa7baa4435c19"} Jul 24 13:18:01.336836 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:01.336801 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:18:01.337100 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:01.337076 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kcjwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 24 13:18:01.337163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:01.337129 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" podUID="b7de505d-0a1c-4d53-9bc6-79eea2973b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 24 13:18:02.336753 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:02.336718 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kcjwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 24 13:18:02.337130 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:02.336785 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" podUID="b7de505d-0a1c-4d53-9bc6-79eea2973b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 24 13:18:03.337244 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:03.337209 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kcjwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 24 13:18:03.337643 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:03.337262 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" podUID="b7de505d-0a1c-4d53-9bc6-79eea2973b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 24 13:18:04.337563 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:04.337505 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-kcjwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 24 13:18:04.337934 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:04.337588 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" podUID="b7de505d-0a1c-4d53-9bc6-79eea2973b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 24 13:18:05.340929 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:05.340900 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:18:05.341318 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:05.341280 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:18:05.341853 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:05.341834 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-kcjwr" Jul 24 13:18:08.227504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:08.227475 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-v4mkx" Jul 24 13:18:08.227880 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:08.227705 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-4tzvz" Jul 24 13:18:39.715523 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.715447 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6"] Jul 24 13:18:39.719152 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.719132 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.722678 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.722653 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 13:18:39.722678 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.722666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4v65n\"" Jul 24 13:18:39.722880 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.722675 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 13:18:39.722880 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.722728 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 24 13:18:39.729390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.729371 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6"] Jul 24 13:18:39.799651 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.799622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqk4j\" (UniqueName: \"kubernetes.io/projected/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-kube-api-access-zqk4j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.799651 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.799655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.799867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.799684 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.799867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.799766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.799867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.799838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.799982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.799872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900453 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900513 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqk4j\" (UniqueName: \"kubernetes.io/projected/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-kube-api-access-zqk4j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900534 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.900974 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.901024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900970 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.901024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.900987 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.902870 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.902843 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.903015 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.903000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:39.908802 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:39.908781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqk4j\" (UniqueName: \"kubernetes.io/projected/70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb-kube-api-access-zqk4j\") pod \"e2e-trlp-test-simulated-kserve-598466548d-qclg6\" (UID: \"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:40.029850 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:40.029781 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:40.158687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:40.158662 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6"] Jul 24 13:18:40.160591 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:18:40.160566 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c03d51_9e6b_43ae_9289_3cfc6a2c0eeb.slice/crio-343e5ba696a575bd746302e564dc93989f84151bafc42c09c84d17bce331115a WatchSource:0}: Error finding container 343e5ba696a575bd746302e564dc93989f84151bafc42c09c84d17bce331115a: Status 404 returned error can't find the container with id 343e5ba696a575bd746302e564dc93989f84151bafc42c09c84d17bce331115a Jul 24 13:18:40.449857 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:40.449819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" event={"ID":"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb","Type":"ContainerStarted","Data":"343e5ba696a575bd746302e564dc93989f84151bafc42c09c84d17bce331115a"} Jul 24 13:18:41.456092 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:41.456056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" event={"ID":"70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb","Type":"ContainerStarted","Data":"3d5eb4221775fd85048da727629ce8a4113109a82dd92f75bdd72f358f97d466"} Jul 24 13:18:41.474749 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:41.474703 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" podStartSLOduration=1.505354257 podStartE2EDuration="2.474688481s" totalImagesPullingTime="969.334224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:18:41.47344176 +0000 UTC m=+548.072686984" watchObservedRunningTime="2026-07-24 13:18:41.474688481 +0000 UTC m=+548.073933691" Jul 24 13:18:50.030089 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:50.030060 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:50.030662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:50.030113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:50.042766 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:50.042745 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:18:50.509395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:18:50.509364 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-qclg6" Jul 24 13:19:10.695846 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.695813 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh"] Jul 24 13:19:10.700951 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.700930 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.703134 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.703109 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 24 13:19:10.710016 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.709997 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh"] Jul 24 13:19:10.773059 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.773029 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgprr\" (UniqueName: \"kubernetes.io/projected/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-kube-api-access-tgprr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.773193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.773069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.773193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.773147 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.773193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.773173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.773193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.773193 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.773328 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.773244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874428 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874628 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgprr\" (UniqueName: \"kubernetes.io/projected/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-kube-api-access-tgprr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874628 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874628 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874628 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874525 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874628 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.874895 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874879 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.875010 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874906 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.875010 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.874941 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.876860 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.876838 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.877093 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.877073 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:10.881809 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:10.881786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgprr\" (UniqueName: \"kubernetes.io/projected/52e9ef94-e95a-4a9d-871e-da1ddce29ca6-kube-api-access-tgprr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v8lfh\" (UID: \"52e9ef94-e95a-4a9d-871e-da1ddce29ca6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:11.013042 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:11.012970 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:11.144903 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:11.144879 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh"] Jul 24 13:19:11.146536 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:19:11.146509 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e9ef94_e95a_4a9d_871e_da1ddce29ca6.slice/crio-5f7ad246eea1c39a06a66362006c6943bdcb81952f77ec6552a8b499a530fa56 WatchSource:0}: Error finding container 5f7ad246eea1c39a06a66362006c6943bdcb81952f77ec6552a8b499a530fa56: Status 404 returned error can't find the container with id 5f7ad246eea1c39a06a66362006c6943bdcb81952f77ec6552a8b499a530fa56 Jul 24 13:19:11.602107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:11.602017 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" event={"ID":"52e9ef94-e95a-4a9d-871e-da1ddce29ca6","Type":"ContainerStarted","Data":"62b1012e1dfd29aa4d3ffdba20b98d048867f618af606a355490976e07cb2396"} Jul 24 13:19:11.602107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:11.602074 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" event={"ID":"52e9ef94-e95a-4a9d-871e-da1ddce29ca6","Type":"ContainerStarted","Data":"5f7ad246eea1c39a06a66362006c6943bdcb81952f77ec6552a8b499a530fa56"} Jul 24 13:19:11.619948 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:11.619901 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" podStartSLOduration=1.431468102 podStartE2EDuration="1.619888086s" totalImagesPullingTime="188.419984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:19:11.61725594 +0000 UTC m=+578.216501152" watchObservedRunningTime="2026-07-24 13:19:11.619888086 +0000 UTC m=+578.219133297" Jul 24 13:19:21.013557 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:21.013517 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:21.013932 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:21.013608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:21.025816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:21.025788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:19:21.660875 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:19:21.660849 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v8lfh" Jul 24 13:21:15.813971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:15.813937 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5655576-n2xbm"] Jul 24 13:21:15.814481 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:15.814139 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf5655576-n2xbm" podUID="a50c8166-01a6-4d61-99c6-f37d82844faa" containerName="manager" containerID="cri-o://86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43" gracePeriod=30 Jul 24 13:21:16.054047 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.054024 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:21:16.130015 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.129936 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68qf4\" (UniqueName: \"kubernetes.io/projected/a50c8166-01a6-4d61-99c6-f37d82844faa-kube-api-access-68qf4\") pod \"a50c8166-01a6-4d61-99c6-f37d82844faa\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " Jul 24 13:21:16.130015 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.129991 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a50c8166-01a6-4d61-99c6-f37d82844faa-webhook-cert\") pod \"a50c8166-01a6-4d61-99c6-f37d82844faa\" (UID: \"a50c8166-01a6-4d61-99c6-f37d82844faa\") " Jul 24 13:21:16.131933 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.131900 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50c8166-01a6-4d61-99c6-f37d82844faa-kube-api-access-68qf4" (OuterVolumeSpecName: "kube-api-access-68qf4") pod "a50c8166-01a6-4d61-99c6-f37d82844faa" (UID: "a50c8166-01a6-4d61-99c6-f37d82844faa"). InnerVolumeSpecName "kube-api-access-68qf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:21:16.132032 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.131931 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50c8166-01a6-4d61-99c6-f37d82844faa-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a50c8166-01a6-4d61-99c6-f37d82844faa" (UID: "a50c8166-01a6-4d61-99c6-f37d82844faa"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:21:16.207342 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.207308 2562 generic.go:356] "Generic (PLEG): container finished" podID="a50c8166-01a6-4d61-99c6-f37d82844faa" containerID="86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43" exitCode=0 Jul 24 13:21:16.207490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.207352 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-n2xbm" Jul 24 13:21:16.207490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.207360 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-n2xbm" event={"ID":"a50c8166-01a6-4d61-99c6-f37d82844faa","Type":"ContainerDied","Data":"86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43"} Jul 24 13:21:16.207490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.207401 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-n2xbm" event={"ID":"a50c8166-01a6-4d61-99c6-f37d82844faa","Type":"ContainerDied","Data":"3fd751796f87334eae0b60e549768e3aef357b4d2a7879a244f3a35357232309"} Jul 24 13:21:16.207490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.207422 2562 scope.go:117] "RemoveContainer" containerID="86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43" Jul 24 13:21:16.217013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.216989 2562 scope.go:117] "RemoveContainer" containerID="86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43" Jul 24 13:21:16.217288 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:21:16.217268 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43\": container with ID starting with 86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43 not found: ID does not exist" containerID="86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43" Jul 24 13:21:16.217353 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.217295 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43"} err="failed to get container status \"86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43\": rpc error: code = NotFound desc = could not find container \"86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43\": container with ID starting with 86a56175066023dd53d4023b5b676caa2e4929bc7f28f04e757a95f53e48ff43 not found: ID does not exist" Jul 24 13:21:16.228712 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.228685 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5655576-n2xbm"] Jul 24 13:21:16.231409 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.231387 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68qf4\" (UniqueName: \"kubernetes.io/projected/a50c8166-01a6-4d61-99c6-f37d82844faa-kube-api-access-68qf4\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:21:16.231495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.231410 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a50c8166-01a6-4d61-99c6-f37d82844faa-webhook-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:21:16.232119 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:16.232102 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf5655576-n2xbm"] Jul 24 13:21:17.895985 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:21:17.895953 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a50c8166-01a6-4d61-99c6-f37d82844faa" path="/var/lib/kubelet/pods/a50c8166-01a6-4d61-99c6-f37d82844faa/volumes" Jul 24 13:22:17.091781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.091749 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf5655576-bc56c"] Jul 24 13:22:17.092315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.092076 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50c8166-01a6-4d61-99c6-f37d82844faa" containerName="manager" Jul 24 13:22:17.092315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.092086 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50c8166-01a6-4d61-99c6-f37d82844faa" containerName="manager" Jul 24 13:22:17.092315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.092152 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50c8166-01a6-4d61-99c6-f37d82844faa" containerName="manager" Jul 24 13:22:17.095082 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.095067 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.098154 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.098134 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:22:17.098283 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.098157 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nbq5n\"" Jul 24 13:22:17.098283 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.098191 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:22:17.103934 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.103914 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5655576-bc56c"] Jul 24 13:22:17.253399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.253375 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpw44\" (UniqueName: \"kubernetes.io/projected/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-kube-api-access-dpw44\") pod \"maas-controller-6cf5655576-bc56c\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.253515 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.253418 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-webhook-cert\") pod \"maas-controller-6cf5655576-bc56c\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.353910 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.353841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpw44\" (UniqueName: \"kubernetes.io/projected/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-kube-api-access-dpw44\") pod \"maas-controller-6cf5655576-bc56c\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.353910 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.353891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-webhook-cert\") pod \"maas-controller-6cf5655576-bc56c\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.356201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.356182 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-webhook-cert\") pod \"maas-controller-6cf5655576-bc56c\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.361956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.361930 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpw44\" (UniqueName: \"kubernetes.io/projected/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-kube-api-access-dpw44\") pod \"maas-controller-6cf5655576-bc56c\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.405719 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.405694 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:17.534507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.534484 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5655576-bc56c"] Jul 24 13:22:17.536413 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:22:17.536380 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a2896c_2b82_43a7_9ce6_9621c51b7ba3.slice/crio-36cbaa965d58820049529a711c13d050c2efcc18d68342072aa715f8568ab073 WatchSource:0}: Error finding container 36cbaa965d58820049529a711c13d050c2efcc18d68342072aa715f8568ab073: Status 404 returned error can't find the container with id 36cbaa965d58820049529a711c13d050c2efcc18d68342072aa715f8568ab073 Jul 24 13:22:17.537665 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:17.537646 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:22:18.522327 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:18.522287 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-bc56c" event={"ID":"66a2896c-2b82-43a7-9ce6-9621c51b7ba3","Type":"ContainerStarted","Data":"36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9"} Jul 24 13:22:18.522709 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:18.522351 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-bc56c" event={"ID":"66a2896c-2b82-43a7-9ce6-9621c51b7ba3","Type":"ContainerStarted","Data":"36cbaa965d58820049529a711c13d050c2efcc18d68342072aa715f8568ab073"} Jul 24 13:22:18.522709 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:18.522383 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:18.539199 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:18.539149 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf5655576-bc56c" podStartSLOduration=1.07681716 podStartE2EDuration="1.539130567s" totalImagesPullingTime="462.313407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:22:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:22:18.538407669 +0000 UTC m=+765.137652882" watchObservedRunningTime="2026-07-24 13:22:18.539130567 +0000 UTC m=+765.138375778" Jul 24 13:22:29.529778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:29.529748 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:34.215301 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.215254 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5655576-bc56c"] Jul 24 13:22:34.215801 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.215504 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf5655576-bc56c" podUID="66a2896c-2b82-43a7-9ce6-9621c51b7ba3" containerName="manager" containerID="cri-o://36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9" gracePeriod=30 Jul 24 13:22:34.456329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.456303 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:34.505088 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.505012 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-webhook-cert\") pod \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " Jul 24 13:22:34.505088 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.505045 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpw44\" (UniqueName: \"kubernetes.io/projected/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-kube-api-access-dpw44\") pod \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\" (UID: \"66a2896c-2b82-43a7-9ce6-9621c51b7ba3\") " Jul 24 13:22:34.507042 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.507015 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "66a2896c-2b82-43a7-9ce6-9621c51b7ba3" (UID: "66a2896c-2b82-43a7-9ce6-9621c51b7ba3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:22:34.507195 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.507175 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-kube-api-access-dpw44" (OuterVolumeSpecName: "kube-api-access-dpw44") pod "66a2896c-2b82-43a7-9ce6-9621c51b7ba3" (UID: "66a2896c-2b82-43a7-9ce6-9621c51b7ba3"). InnerVolumeSpecName "kube-api-access-dpw44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:22:34.601764 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.601734 2562 generic.go:356] "Generic (PLEG): container finished" podID="66a2896c-2b82-43a7-9ce6-9621c51b7ba3" containerID="36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9" exitCode=0 Jul 24 13:22:34.601890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.601781 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-bc56c" Jul 24 13:22:34.601890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.601825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-bc56c" event={"ID":"66a2896c-2b82-43a7-9ce6-9621c51b7ba3","Type":"ContainerDied","Data":"36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9"} Jul 24 13:22:34.601890 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.601882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-bc56c" event={"ID":"66a2896c-2b82-43a7-9ce6-9621c51b7ba3","Type":"ContainerDied","Data":"36cbaa965d58820049529a711c13d050c2efcc18d68342072aa715f8568ab073"} Jul 24 13:22:34.601983 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.601909 2562 scope.go:117] "RemoveContainer" containerID="36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9" Jul 24 13:22:34.606229 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.606209 2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-webhook-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:22:34.606328 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.606232 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpw44\" (UniqueName: \"kubernetes.io/projected/66a2896c-2b82-43a7-9ce6-9621c51b7ba3-kube-api-access-dpw44\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:22:34.610495 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.610481 2562 scope.go:117] "RemoveContainer" containerID="36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9" Jul 24 13:22:34.610791 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:22:34.610773 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9\": container with ID starting with 36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9 not found: ID does not exist" containerID="36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9" Jul 24 13:22:34.610861 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.610795 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9"} err="failed to get container status \"36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9\": rpc error: code = NotFound desc = could not find container \"36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9\": container with ID starting with 36157e6836bd2964c5856eab6c4ce71093de3e24a831c15ca7318f6f1942e8e9 not found: ID does not exist" Jul 24 13:22:34.622171 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.622151 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf5655576-bc56c"] Jul 24 13:22:34.625344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:34.625324 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf5655576-bc56c"] Jul 24 13:22:35.455724 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.455695 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf5655576-4xrjx"] Jul 24 13:22:35.456118 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.456043 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a2896c-2b82-43a7-9ce6-9621c51b7ba3" containerName="manager" Jul 24 13:22:35.456118 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.456054 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a2896c-2b82-43a7-9ce6-9621c51b7ba3" containerName="manager" Jul 24 13:22:35.456118 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.456115 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a2896c-2b82-43a7-9ce6-9621c51b7ba3" containerName="manager" Jul 24 13:22:35.460101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.460082 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.463196 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.463175 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 24 13:22:35.463302 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.463179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nbq5n\"" Jul 24 13:22:35.463302 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.463183 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 24 13:22:35.468482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.468462 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5655576-4xrjx"] Jul 24 13:22:35.512847 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.512820 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrd4k\" (UniqueName: \"kubernetes.io/projected/260770a9-bdf6-4a31-b462-ae4f4f3b0d97-kube-api-access-mrd4k\") pod \"maas-controller-6cf5655576-4xrjx\" (UID: \"260770a9-bdf6-4a31-b462-ae4f4f3b0d97\") " pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.512971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.512852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/260770a9-bdf6-4a31-b462-ae4f4f3b0d97-webhook-cert\") pod \"maas-controller-6cf5655576-4xrjx\" (UID: \"260770a9-bdf6-4a31-b462-ae4f4f3b0d97\") " pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.613903 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.613880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrd4k\" (UniqueName: \"kubernetes.io/projected/260770a9-bdf6-4a31-b462-ae4f4f3b0d97-kube-api-access-mrd4k\") pod \"maas-controller-6cf5655576-4xrjx\" (UID: \"260770a9-bdf6-4a31-b462-ae4f4f3b0d97\") " pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.614046 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.613914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/260770a9-bdf6-4a31-b462-ae4f4f3b0d97-webhook-cert\") pod \"maas-controller-6cf5655576-4xrjx\" (UID: \"260770a9-bdf6-4a31-b462-ae4f4f3b0d97\") " pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.616293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.616270 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/260770a9-bdf6-4a31-b462-ae4f4f3b0d97-webhook-cert\") pod \"maas-controller-6cf5655576-4xrjx\" (UID: \"260770a9-bdf6-4a31-b462-ae4f4f3b0d97\") " pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.622255 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.622235 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrd4k\" (UniqueName: \"kubernetes.io/projected/260770a9-bdf6-4a31-b462-ae4f4f3b0d97-kube-api-access-mrd4k\") pod \"maas-controller-6cf5655576-4xrjx\" (UID: \"260770a9-bdf6-4a31-b462-ae4f4f3b0d97\") " pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.771709 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.771632 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:35.897949 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.897921 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a2896c-2b82-43a7-9ce6-9621c51b7ba3" path="/var/lib/kubelet/pods/66a2896c-2b82-43a7-9ce6-9621c51b7ba3/volumes" Jul 24 13:22:35.909277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:35.909253 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf5655576-4xrjx"] Jul 24 13:22:35.910978 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:22:35.910954 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260770a9_bdf6_4a31_b462_ae4f4f3b0d97.slice/crio-a0fcdaf1a05e17ea5688cc88e042ed9d3c35137509d5bfc1a59acd25edb8cd09 WatchSource:0}: Error finding container a0fcdaf1a05e17ea5688cc88e042ed9d3c35137509d5bfc1a59acd25edb8cd09: Status 404 returned error can't find the container with id a0fcdaf1a05e17ea5688cc88e042ed9d3c35137509d5bfc1a59acd25edb8cd09 Jul 24 13:22:36.615299 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:36.615208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-4xrjx" event={"ID":"260770a9-bdf6-4a31-b462-ae4f4f3b0d97","Type":"ContainerStarted","Data":"66097f3e76ffca212e8abd98a645527a0bab9d80f2dd8359eba4f8020c0788cf"} Jul 24 13:22:36.615686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:36.615306 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf5655576-4xrjx" event={"ID":"260770a9-bdf6-4a31-b462-ae4f4f3b0d97","Type":"ContainerStarted","Data":"a0fcdaf1a05e17ea5688cc88e042ed9d3c35137509d5bfc1a59acd25edb8cd09"} Jul 24 13:22:36.615686 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:36.615336 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:22:36.633123 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:36.633066 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf5655576-4xrjx" podStartSLOduration=1.202176792 podStartE2EDuration="1.633050055s" totalImagesPullingTime="430.873263ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:22:36.631178624 +0000 UTC m=+783.230423835" watchObservedRunningTime="2026-07-24 13:22:36.633050055 +0000 UTC m=+783.232295265" Jul 24 13:22:47.622403 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:22:47.622372 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf5655576-4xrjx" Jul 24 13:31:28.215963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.215933 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79"] Jul 24 13:31:28.219192 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.219171 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.221458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.221436 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-c82697db-kserve-self-signed-certs\"" Jul 24 13:31:28.231617 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.231594 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79"] Jul 24 13:31:28.279611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.279579 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbvx7\" (UniqueName: \"kubernetes.io/projected/d05a0b05-2693-4ad1-8823-15518a28c9a7-kube-api-access-tbvx7\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.279767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.279622 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05a0b05-2693-4ad1-8823-15518a28c9a7-tls-certs\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.279767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.279712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-home\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.279767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.279760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-tmp-dir\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.279914 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.279828 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-model-cache\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.279914 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.279875 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-dshm\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.380898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.380862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-home\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.380898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.380898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-tmp-dir\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.380930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-model-cache\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.380955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-dshm\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.381002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbvx7\" (UniqueName: \"kubernetes.io/projected/d05a0b05-2693-4ad1-8823-15518a28c9a7-kube-api-access-tbvx7\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.381025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05a0b05-2693-4ad1-8823-15518a28c9a7-tls-certs\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381355 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.381326 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-tmp-dir\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381355 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.381338 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-model-cache\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.381454 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.381390 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-home\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.383348 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.383325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-dshm\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.383564 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.383522 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05a0b05-2693-4ad1-8823-15518a28c9a7-tls-certs\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.389766 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.389716 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbvx7\" (UniqueName: \"kubernetes.io/projected/d05a0b05-2693-4ad1-8823-15518a28c9a7-kube-api-access-tbvx7\") pod \"e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.528090 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.528010 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:28.662086 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:28.662061 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79"] Jul 24 13:31:28.663765 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:31:28.663737 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd05a0b05_2693_4ad1_8823_15518a28c9a7.slice/crio-e7b6c16df1abf2f7970e40210a945f76d5ceb5380c42d1642ed6314247ebc0cd WatchSource:0}: Error finding container e7b6c16df1abf2f7970e40210a945f76d5ceb5380c42d1642ed6314247ebc0cd: Status 404 returned error can't find the container with id e7b6c16df1abf2f7970e40210a945f76d5ceb5380c42d1642ed6314247ebc0cd Jul 24 13:31:29.196199 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:29.196161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" event={"ID":"d05a0b05-2693-4ad1-8823-15518a28c9a7","Type":"ContainerStarted","Data":"c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d"} Jul 24 13:31:29.196398 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:29.196211 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" event={"ID":"d05a0b05-2693-4ad1-8823-15518a28c9a7","Type":"ContainerStarted","Data":"e7b6c16df1abf2f7970e40210a945f76d5ceb5380c42d1642ed6314247ebc0cd"} Jul 24 13:31:29.218603 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:29.217344 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" podStartSLOduration=1.21732285 podStartE2EDuration="1.21732285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:31:29.212826581 +0000 UTC m=+1315.812071804" watchObservedRunningTime="2026-07-24 13:31:29.21732285 +0000 UTC m=+1315.816568062" Jul 24 13:31:38.528524 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:38.528478 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:38.529045 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:38.528572 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:38.541460 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:38.541433 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:31:39.257204 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:31:39.257178 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:32:01.982137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:01.982059 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79"] Jul 24 13:32:17.001193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.001143 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" podUID="d05a0b05-2693-4ad1-8823-15518a28c9a7" containerName="main" containerID="cri-o://c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d" gracePeriod=45 Jul 24 13:32:17.252981 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.252916 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:32:17.285142 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285113 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbvx7\" (UniqueName: \"kubernetes.io/projected/d05a0b05-2693-4ad1-8823-15518a28c9a7-kube-api-access-tbvx7\") pod \"d05a0b05-2693-4ad1-8823-15518a28c9a7\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " Jul 24 13:32:17.285275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285157 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-dshm\") pod \"d05a0b05-2693-4ad1-8823-15518a28c9a7\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " Jul 24 13:32:17.285275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285187 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-tmp-dir\") pod \"d05a0b05-2693-4ad1-8823-15518a28c9a7\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " Jul 24 13:32:17.285275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285214 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-model-cache\") pod \"d05a0b05-2693-4ad1-8823-15518a28c9a7\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " Jul 24 13:32:17.285440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285275 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-home\") pod \"d05a0b05-2693-4ad1-8823-15518a28c9a7\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " Jul 24 13:32:17.285440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285320 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05a0b05-2693-4ad1-8823-15518a28c9a7-tls-certs\") pod \"d05a0b05-2693-4ad1-8823-15518a28c9a7\" (UID: \"d05a0b05-2693-4ad1-8823-15518a28c9a7\") " Jul 24 13:32:17.285557 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285505 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d05a0b05-2693-4ad1-8823-15518a28c9a7" (UID: "d05a0b05-2693-4ad1-8823-15518a28c9a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:32:17.285557 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285516 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "d05a0b05-2693-4ad1-8823-15518a28c9a7" (UID: "d05a0b05-2693-4ad1-8823-15518a28c9a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:32:17.285657 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285570 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-home" (OuterVolumeSpecName: "home") pod "d05a0b05-2693-4ad1-8823-15518a28c9a7" (UID: "d05a0b05-2693-4ad1-8823-15518a28c9a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:32:17.285697 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285682 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-tmp-dir\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:32:17.285731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285703 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-model-cache\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:32:17.285731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.285716 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-home\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:32:17.287399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.287374 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-dshm" (OuterVolumeSpecName: "dshm") pod "d05a0b05-2693-4ad1-8823-15518a28c9a7" (UID: "d05a0b05-2693-4ad1-8823-15518a28c9a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:32:17.287501 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.287421 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d05a0b05-2693-4ad1-8823-15518a28c9a7-kube-api-access-tbvx7" (OuterVolumeSpecName: "kube-api-access-tbvx7") pod "d05a0b05-2693-4ad1-8823-15518a28c9a7" (UID: "d05a0b05-2693-4ad1-8823-15518a28c9a7"). InnerVolumeSpecName "kube-api-access-tbvx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:32:17.287501 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.287433 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d05a0b05-2693-4ad1-8823-15518a28c9a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d05a0b05-2693-4ad1-8823-15518a28c9a7" (UID: "d05a0b05-2693-4ad1-8823-15518a28c9a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:32:17.386935 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.386905 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbvx7\" (UniqueName: \"kubernetes.io/projected/d05a0b05-2693-4ad1-8823-15518a28c9a7-kube-api-access-tbvx7\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:32:17.386935 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.386930 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d05a0b05-2693-4ad1-8823-15518a28c9a7-dshm\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:32:17.386935 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.386940 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d05a0b05-2693-4ad1-8823-15518a28c9a7-tls-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:32:17.435371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.435340 2562 generic.go:356] "Generic (PLEG): container finished" podID="d05a0b05-2693-4ad1-8823-15518a28c9a7" containerID="c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d" exitCode=0 Jul 24 13:32:17.435534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.435390 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" Jul 24 13:32:17.435534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.435428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" event={"ID":"d05a0b05-2693-4ad1-8823-15518a28c9a7","Type":"ContainerDied","Data":"c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d"} Jul 24 13:32:17.435534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.435480 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79" event={"ID":"d05a0b05-2693-4ad1-8823-15518a28c9a7","Type":"ContainerDied","Data":"e7b6c16df1abf2f7970e40210a945f76d5ceb5380c42d1642ed6314247ebc0cd"} Jul 24 13:32:17.435534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.435502 2562 scope.go:117] "RemoveContainer" containerID="c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d" Jul 24 13:32:17.444949 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.444934 2562 scope.go:117] "RemoveContainer" containerID="c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d" Jul 24 13:32:17.445202 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:32:17.445180 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d\": container with ID starting with c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d not found: ID does not exist" containerID="c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d" Jul 24 13:32:17.445277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.445213 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d"} err="failed to get container status \"c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d\": rpc error: code = NotFound desc = could not find container \"c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d\": container with ID starting with c6a0599f33f5fe7356b0140f234d853834c6c070807e487fc81c800c9e2d055d not found: ID does not exist" Jul 24 13:32:17.459006 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.458985 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79"] Jul 24 13:32:17.468266 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.468246 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-c82697db-kserve-85d74499c5-8fj79"] Jul 24 13:32:17.896627 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:32:17.896594 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d05a0b05-2693-4ad1-8823-15518a28c9a7" path="/var/lib/kubelet/pods/d05a0b05-2693-4ad1-8823-15518a28c9a7/volumes" Jul 24 13:35:45.022422 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.022389 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm"] Jul 24 13:35:45.024864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.022849 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d05a0b05-2693-4ad1-8823-15518a28c9a7" containerName="main" Jul 24 13:35:45.024864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.022867 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d05a0b05-2693-4ad1-8823-15518a28c9a7" containerName="main" Jul 24 13:35:45.024864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.022962 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d05a0b05-2693-4ad1-8823-15518a28c9a7" containerName="main" Jul 24 13:35:45.025798 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.025785 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.027965 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.027944 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-77101517-openshift-default-dockercfg-kkmft\"" Jul 24 13:35:45.054334 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.054309 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm"] Jul 24 13:35:45.057658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057637 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7lzv\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-kube-api-access-v7lzv\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.057738 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057674 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf4baf3-9c62-4067-80ce-be4bc915a082-istiod-ca-cert\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.057815 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-socket\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.057851 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057842 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-certs\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.057884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057866 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-envoy\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.057927 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-data\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.058000 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.057979 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-podinfo\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.058056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.058022 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-credential-socket\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.058148 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.058121 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-token\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159172 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159142 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-socket\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159172 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-certs\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159193 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-envoy\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-data\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-podinfo\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-credential-socket\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-token\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159362 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7lzv\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-kube-api-access-v7lzv\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159416 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf4baf3-9c62-4067-80ce-be4bc915a082-istiod-ca-cert\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159802 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-data\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159802 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159691 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-certs\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159802 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159747 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-socket\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.159963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.159952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-credential-socket\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.160158 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.160137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf4baf3-9c62-4067-80ce-be4bc915a082-istiod-ca-cert\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.161454 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.161433 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-envoy\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.161706 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.161689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-podinfo\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.175372 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.175350 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7lzv\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-kube-api-access-v7lzv\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.176480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.176459 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-token\") pod \"e2e-ait-77101517-openshift-default-7878bb577d-4fwjm\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.336051 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.335966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:45.476760 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.476736 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm"] Jul 24 13:35:45.478333 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:35:45.478304 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf4baf3_9c62_4067_80ce_be4bc915a082.slice/crio-c882ba1afd24c83c4889edad4a5891017bc817282be5ae9a2f541da62f3a89bf WatchSource:0}: Error finding container c882ba1afd24c83c4889edad4a5891017bc817282be5ae9a2f541da62f3a89bf: Status 404 returned error can't find the container with id c882ba1afd24c83c4889edad4a5891017bc817282be5ae9a2f541da62f3a89bf Jul 24 13:35:45.480383 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.480356 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:35:45.480444 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.480415 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:35:45.480480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:45.480443 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:35:46.469425 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:46.469394 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" event={"ID":"ebf4baf3-9c62-4067-80ce-be4bc915a082","Type":"ContainerStarted","Data":"e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e"} Jul 24 13:35:46.469814 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:46.469447 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" event={"ID":"ebf4baf3-9c62-4067-80ce-be4bc915a082","Type":"ContainerStarted","Data":"c882ba1afd24c83c4889edad4a5891017bc817282be5ae9a2f541da62f3a89bf"} Jul 24 13:35:46.506698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:46.506648 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podStartSLOduration=2.506630791 podStartE2EDuration="2.506630791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:46.503743357 +0000 UTC m=+1573.102988570" watchObservedRunningTime="2026-07-24 13:35:46.506630791 +0000 UTC m=+1573.105876072" Jul 24 13:35:47.336153 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:47.336115 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:47.340995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:47.340967 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:47.474313 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:47.474283 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:47.475113 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:47.475094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:35:48.096007 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.095972 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp"] Jul 24 13:35:48.099537 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.099506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.105653 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.105630 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h9lpb\"" Jul 24 13:35:48.105781 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.105631 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-77101517-serving-cert\"" Jul 24 13:35:48.109048 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.109023 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp"] Jul 24 13:35:48.185301 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.185275 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-maas-api-tls\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.185454 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.185307 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8nnh\" (UniqueName: \"kubernetes.io/projected/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-kube-api-access-s8nnh\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.185454 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.185391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-openshift-service-ca\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.286877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.286845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-openshift-service-ca\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.287041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.286923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-maas-api-tls\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.287041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.286942 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8nnh\" (UniqueName: \"kubernetes.io/projected/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-kube-api-access-s8nnh\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.287570 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.287518 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-openshift-service-ca\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.289348 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.289328 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-maas-api-tls\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.295799 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.295755 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8nnh\" (UniqueName: \"kubernetes.io/projected/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-kube-api-access-s8nnh\") pod \"maas-api-e2e-ait-77101517-74bc97cbb9-snfqp\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.411937 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.411856 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:48.552536 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.552508 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp"] Jul 24 13:35:48.554188 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:35:48.554162 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae36582c_f62a_4a77_8e26_8ad7a17fe72e.slice/crio-1f93d5b5dc366bdde16cca17cba31d5585c6f42c9ef5e0aef742316fb191e6a2 WatchSource:0}: Error finding container 1f93d5b5dc366bdde16cca17cba31d5585c6f42c9ef5e0aef742316fb191e6a2: Status 404 returned error can't find the container with id 1f93d5b5dc366bdde16cca17cba31d5585c6f42c9ef5e0aef742316fb191e6a2 Jul 24 13:35:48.555504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:48.555482 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:35:49.488225 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:49.488195 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" event={"ID":"ae36582c-f62a-4a77-8e26-8ad7a17fe72e","Type":"ContainerStarted","Data":"3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f"} Jul 24 13:35:49.488388 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:49.488253 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" event={"ID":"ae36582c-f62a-4a77-8e26-8ad7a17fe72e","Type":"ContainerStarted","Data":"1f93d5b5dc366bdde16cca17cba31d5585c6f42c9ef5e0aef742316fb191e6a2"} Jul 24 13:35:49.507273 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:49.507224 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" podStartSLOduration=0.947310389 podStartE2EDuration="1.507208724s" totalImagesPullingTime="559.898335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:35:49.504886514 +0000 UTC m=+1576.104131750" watchObservedRunningTime="2026-07-24 13:35:49.507208724 +0000 UTC m=+1576.106453936" Jul 24 13:35:50.493785 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:50.493754 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:35:56.510622 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:35:56.510589 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:36:02.894767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:02.893823 2562 patch_prober.go:28] interesting pod/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:48664->10.133.0.43:15021: read: connection reset by peer" start-of-body= Jul 24 13:36:02.894767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:02.893915 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": read tcp 10.133.0.2:48664->10.133.0.43:15021: read: connection reset by peer" Jul 24 13:36:03.561524 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:03.561492 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerID="e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e" exitCode=0 Jul 24 13:36:03.561710 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:03.561573 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" event={"ID":"ebf4baf3-9c62-4067-80ce-be4bc915a082","Type":"ContainerDied","Data":"e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e"} Jul 24 13:36:03.561973 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:03.561960 2562 scope.go:117] "RemoveContainer" containerID="e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e" Jul 24 13:36:03.562940 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:03.562913 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:03.563040 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:03.563018 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:03.563101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:03.563067 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:04.568696 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:04.568665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" event={"ID":"ebf4baf3-9c62-4067-80ce-be4bc915a082","Type":"ContainerStarted","Data":"a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14"} Jul 24 13:36:05.337191 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:05.337149 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:36:05.337403 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:05.337380 2562 patch_prober.go:28] interesting pod/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 24 13:36:05.337459 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:05.337436 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 24 13:36:06.337068 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:06.337034 2562 patch_prober.go:28] interesting pod/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 24 13:36:06.337514 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:06.337092 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 24 13:36:07.336962 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:07.336928 2562 patch_prober.go:28] interesting pod/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 24 13:36:07.337139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:07.336981 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 24 13:36:08.336970 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:08.336933 2562 patch_prober.go:28] interesting pod/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 24 13:36:08.337143 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:08.336991 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 24 13:36:09.339540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:09.339505 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:36:09.340023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:09.340000 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:36:09.340628 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:09.340611 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:36:24.002441 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.002397 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp"] Jul 24 13:36:24.003056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.002689 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" podUID="ae36582c-f62a-4a77-8e26-8ad7a17fe72e" containerName="maas-api" containerID="cri-o://3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f" gracePeriod=30 Jul 24 13:36:24.269138 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.269105 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:36:24.313633 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.313602 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8nnh\" (UniqueName: \"kubernetes.io/projected/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-kube-api-access-s8nnh\") pod \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " Jul 24 13:36:24.313800 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.313638 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-openshift-service-ca\") pod \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " Jul 24 13:36:24.313800 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.313681 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-maas-api-tls\") pod \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\" (UID: \"ae36582c-f62a-4a77-8e26-8ad7a17fe72e\") " Jul 24 13:36:24.314023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.313995 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae36582c-f62a-4a77-8e26-8ad7a17fe72e" (UID: "ae36582c-f62a-4a77-8e26-8ad7a17fe72e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:36:24.315846 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.315825 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-kube-api-access-s8nnh" (OuterVolumeSpecName: "kube-api-access-s8nnh") pod "ae36582c-f62a-4a77-8e26-8ad7a17fe72e" (UID: "ae36582c-f62a-4a77-8e26-8ad7a17fe72e"). InnerVolumeSpecName "kube-api-access-s8nnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:36:24.315927 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.315848 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae36582c-f62a-4a77-8e26-8ad7a17fe72e" (UID: "ae36582c-f62a-4a77-8e26-8ad7a17fe72e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:36:24.414807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.414775 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8nnh\" (UniqueName: \"kubernetes.io/projected/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-kube-api-access-s8nnh\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:24.414807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.414803 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-openshift-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:24.414972 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.414818 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae36582c-f62a-4a77-8e26-8ad7a17fe72e-maas-api-tls\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:24.673534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.673446 2562 generic.go:356] "Generic (PLEG): container finished" podID="ae36582c-f62a-4a77-8e26-8ad7a17fe72e" containerID="3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f" exitCode=0 Jul 24 13:36:24.673534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.673491 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" Jul 24 13:36:24.673534 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.673518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" event={"ID":"ae36582c-f62a-4a77-8e26-8ad7a17fe72e","Type":"ContainerDied","Data":"3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f"} Jul 24 13:36:24.673896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.673597 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp" event={"ID":"ae36582c-f62a-4a77-8e26-8ad7a17fe72e","Type":"ContainerDied","Data":"1f93d5b5dc366bdde16cca17cba31d5585c6f42c9ef5e0aef742316fb191e6a2"} Jul 24 13:36:24.673896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.673628 2562 scope.go:117] "RemoveContainer" containerID="3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f" Jul 24 13:36:24.687642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.687620 2562 scope.go:117] "RemoveContainer" containerID="3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f" Jul 24 13:36:24.687963 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:36:24.687938 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f\": container with ID starting with 3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f not found: ID does not exist" containerID="3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f" Jul 24 13:36:24.688042 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.687973 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f"} err="failed to get container status \"3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f\": rpc error: code = NotFound desc = could not find container \"3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f\": container with ID starting with 3ab11c6bf9a8b7ce01e247ed6ed862b246d7c1feb4274bb31ed621e5c4e97c2f not found: ID does not exist" Jul 24 13:36:24.699200 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.699177 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp"] Jul 24 13:36:24.704414 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:24.704394 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-77101517-74bc97cbb9-snfqp"] Jul 24 13:36:25.895905 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:25.895835 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae36582c-f62a-4a77-8e26-8ad7a17fe72e" path="/var/lib/kubelet/pods/ae36582c-f62a-4a77-8e26-8ad7a17fe72e/volumes" Jul 24 13:36:28.775273 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:28.775238 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm"] Jul 24 13:36:28.775678 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:28.775479 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" containerID="cri-o://a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14" gracePeriod=30 Jul 24 13:36:29.890293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.890260 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz"] Jul 24 13:36:29.890658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.890637 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae36582c-f62a-4a77-8e26-8ad7a17fe72e" containerName="maas-api" Jul 24 13:36:29.890658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.890649 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae36582c-f62a-4a77-8e26-8ad7a17fe72e" containerName="maas-api" Jul 24 13:36:29.890733 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.890719 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae36582c-f62a-4a77-8e26-8ad7a17fe72e" containerName="maas-api" Jul 24 13:36:29.895297 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.895271 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.898649 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.898630 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-51268fa8-gw-openshift-default-dockercfg-qndff\"" Jul 24 13:36:29.906898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.906877 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz"] Jul 24 13:36:29.964641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.964603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-data\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.964825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.964672 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-credential-socket\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.964825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.964697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-certs\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.964825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.964717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-podinfo\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.965003 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.964826 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-token\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.965003 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.964902 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-envoy\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.965098 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.965004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbqx5\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-kube-api-access-vbqx5\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.965098 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.965037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-socket\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:29.965171 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:29.965104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istiod-ca-cert\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.065902 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.065864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-socket\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.065902 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.065904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istiod-ca-cert\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.065927 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-data\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.065956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-credential-socket\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.065972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-certs\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.065988 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-podinfo\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066016 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-token\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-envoy\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbqx5\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-kube-api-access-vbqx5\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066572 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066345 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-socket\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066572 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-data\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066697 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-credential-socket\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066754 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066717 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istiod-ca-cert\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.066754 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.066741 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-certs\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.068487 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.068464 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-envoy\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.068610 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.068596 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-podinfo\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.074516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.074495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbqx5\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-kube-api-access-vbqx5\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.081097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.081076 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-token\") pod \"e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.206662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.206633 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:30.355193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.355163 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz"] Jul 24 13:36:30.355808 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:36:30.355783 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc063dfb7_bc63_42e7_9453_da9a7fc202d9.slice/crio-cfb885ccf2b221eb4ee2475aa64a1808106dc97ded9ae5a1bdd26236f5ae2fcf WatchSource:0}: Error finding container cfb885ccf2b221eb4ee2475aa64a1808106dc97ded9ae5a1bdd26236f5ae2fcf: Status 404 returned error can't find the container with id cfb885ccf2b221eb4ee2475aa64a1808106dc97ded9ae5a1bdd26236f5ae2fcf Jul 24 13:36:30.357886 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.357855 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:30.357970 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.357913 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:30.357970 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.357943 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:30.708095 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.708066 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" event={"ID":"c063dfb7-bc63-42e7-9453-da9a7fc202d9","Type":"ContainerStarted","Data":"d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d"} Jul 24 13:36:30.708259 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.708116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" event={"ID":"c063dfb7-bc63-42e7-9453-da9a7fc202d9","Type":"ContainerStarted","Data":"cfb885ccf2b221eb4ee2475aa64a1808106dc97ded9ae5a1bdd26236f5ae2fcf"} Jul 24 13:36:30.727196 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:30.727143 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podStartSLOduration=1.727125489 podStartE2EDuration="1.727125489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:36:30.726810648 +0000 UTC m=+1617.326055860" watchObservedRunningTime="2026-07-24 13:36:30.727125489 +0000 UTC m=+1617.326370700" Jul 24 13:36:31.207601 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:31.207578 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:31.212473 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:31.212447 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:31.714691 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:31.714659 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:31.715610 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:31.715591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:34.031944 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.031917 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:36:34.104070 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.103985 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf4baf3-9c62-4067-80ce-be4bc915a082-istiod-ca-cert\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104070 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104019 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-envoy\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104070 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104062 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-token\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104089 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-socket\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104151 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-credential-socket\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104198 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-certs\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104221 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-data\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104315 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104243 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-podinfo\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104318 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7lzv\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-kube-api-access-v7lzv\") pod \"ebf4baf3-9c62-4067-80ce-be4bc915a082\" (UID: \"ebf4baf3-9c62-4067-80ce-be4bc915a082\") " Jul 24 13:36:34.104611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104475 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:34.104611 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104495 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:34.104763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104711 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.104763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104733 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.104763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104750 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebf4baf3-9c62-4067-80ce-be4bc915a082-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:36:34.104921 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.104898 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:34.105140 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.105102 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-data" (OuterVolumeSpecName: "istio-data") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:34.106619 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.106594 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-token" (OuterVolumeSpecName: "istio-token") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:36:34.106772 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.106685 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-kube-api-access-v7lzv" (OuterVolumeSpecName: "kube-api-access-v7lzv") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "kube-api-access-v7lzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:36:34.106837 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.106771 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:36:34.106884 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.106859 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ebf4baf3-9c62-4067-80ce-be4bc915a082" (UID: "ebf4baf3-9c62-4067-80ce-be4bc915a082"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:36:34.205284 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205251 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7lzv\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-kube-api-access-v7lzv\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.205284 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205279 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebf4baf3-9c62-4067-80ce-be4bc915a082-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.205284 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205290 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.205504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205298 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.205504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205306 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.205504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205314 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.205504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.205324 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebf4baf3-9c62-4067-80ce-be4bc915a082-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:36:34.730325 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.730292 2562 generic.go:356] "Generic (PLEG): container finished" podID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerID="a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14" exitCode=0 Jul 24 13:36:34.730507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.730342 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" Jul 24 13:36:34.730507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.730380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" event={"ID":"ebf4baf3-9c62-4067-80ce-be4bc915a082","Type":"ContainerDied","Data":"a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14"} Jul 24 13:36:34.730507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.730436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm" event={"ID":"ebf4baf3-9c62-4067-80ce-be4bc915a082","Type":"ContainerDied","Data":"c882ba1afd24c83c4889edad4a5891017bc817282be5ae9a2f541da62f3a89bf"} Jul 24 13:36:34.730507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.730460 2562 scope.go:117] "RemoveContainer" containerID="a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14" Jul 24 13:36:34.740468 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.740450 2562 scope.go:117] "RemoveContainer" containerID="e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e" Jul 24 13:36:34.748509 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.748494 2562 scope.go:117] "RemoveContainer" containerID="a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14" Jul 24 13:36:34.748777 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:36:34.748759 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14\": container with ID starting with a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14 not found: ID does not exist" containerID="a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14" Jul 24 13:36:34.748849 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.748791 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14"} err="failed to get container status \"a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14\": rpc error: code = NotFound desc = could not find container \"a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14\": container with ID starting with a00174202af50ac0a1867f5c99a0888b6fa196e5c0f488f9a9ec5f6dc0997c14 not found: ID does not exist" Jul 24 13:36:34.748849 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.748816 2562 scope.go:117] "RemoveContainer" containerID="e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e" Jul 24 13:36:34.749079 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:36:34.749062 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e\": container with ID starting with e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e not found: ID does not exist" containerID="e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e" Jul 24 13:36:34.749118 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.749086 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e"} err="failed to get container status \"e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e\": rpc error: code = NotFound desc = could not find container \"e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e\": container with ID starting with e358332462075438fac5a8e16ade4ec827a5a0bf137d29f9b2c27326f8daaa1e not found: ID does not exist" Jul 24 13:36:34.753248 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.753229 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm"] Jul 24 13:36:34.757268 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:34.757238 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-77101517-openshift-default-7878bb577d-4fwjm"] Jul 24 13:36:35.896209 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:35.896170 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" path="/var/lib/kubelet/pods/ebf4baf3-9c62-4067-80ce-be4bc915a082/volumes" Jul 24 13:36:46.716896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.716857 2562 patch_prober.go:28] interesting pod/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 13:36:46.717303 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.716917 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 13:36:46.794253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.794222 2562 generic.go:356] "Generic (PLEG): container finished" podID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerID="d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d" exitCode=0 Jul 24 13:36:46.794418 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.794296 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" event={"ID":"c063dfb7-bc63-42e7-9453-da9a7fc202d9","Type":"ContainerDied","Data":"d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d"} Jul 24 13:36:46.794719 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.794696 2562 scope.go:117] "RemoveContainer" containerID="d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d" Jul 24 13:36:46.795722 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.795696 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:46.795819 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.795779 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:46.795883 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:46.795830 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:36:47.806019 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:47.805973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" event={"ID":"c063dfb7-bc63-42e7-9453-da9a7fc202d9","Type":"ContainerStarted","Data":"b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224"} Jul 24 13:36:48.207236 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:48.207193 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:48.207484 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:48.207457 2562 patch_prober.go:28] interesting pod/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 13:36:48.207588 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:48.207560 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 13:36:49.207523 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:49.207481 2562 patch_prober.go:28] interesting pod/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 13:36:49.208040 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:49.207615 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 13:36:50.208077 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:50.208032 2562 patch_prober.go:28] interesting pod/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 13:36:50.208581 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:50.208105 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 13:36:51.207512 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:51.207466 2562 patch_prober.go:28] interesting pod/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 24 13:36:51.207743 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:51.207539 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 24 13:36:52.211152 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:52.211121 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:52.211528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:52.211408 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:36:52.212075 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:36:52.212057 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:37:12.540394 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:12.540362 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz"] Jul 24 13:37:12.540913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:12.540600 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" containerID="cri-o://b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224" gracePeriod=30 Jul 24 13:37:13.371692 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.371658 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2"] Jul 24 13:37:13.372004 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.371992 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" Jul 24 13:37:13.372048 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.372006 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" Jul 24 13:37:13.372048 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.372018 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" Jul 24 13:37:13.372048 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.372023 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" Jul 24 13:37:13.372149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.372094 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" Jul 24 13:37:13.372149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.372104 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebf4baf3-9c62-4067-80ce-be4bc915a082" containerName="istio-proxy" Jul 24 13:37:13.375117 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.375091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.377770 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.377746 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ca810d9a-openshift-default-dockercfg-8z5tc\"" Jul 24 13:37:13.386813 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.386788 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2"] Jul 24 13:37:13.464564 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-socket\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464745 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-credential-socket\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464745 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-podinfo\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464745 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-certs\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464745 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-token\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464979 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istiod-ca-cert\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464979 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-envoy\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464979 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-data\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.464979 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.464952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnnpl\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-kube-api-access-pnnpl\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.565887 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.565849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-credential-socket\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.565887 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.565890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-podinfo\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.565941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-certs\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.565970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-token\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istiod-ca-cert\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-envoy\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-data\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnnpl\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-kube-api-access-pnnpl\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-socket\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566399 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566330 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-credential-socket\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566784 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566625 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-socket\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566784 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566665 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-certs\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.566784 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.566715 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-data\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.567071 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.567049 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istiod-ca-cert\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.568294 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.568277 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-envoy\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.568513 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.568495 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-podinfo\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.573810 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.573790 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-token\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.574018 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.574000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnnpl\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-kube-api-access-pnnpl\") pod \"e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.686098 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.686071 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:13.827117 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.827070 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2"] Jul 24 13:37:13.829412 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:37:13.829382 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b79f972_f9fd_4f53_98a3_4abdc45a68fc.slice/crio-726d7a711ebc36dd580815fde6974e63ac8e280f1fedb66cd9c59c47566233e4 WatchSource:0}: Error finding container 726d7a711ebc36dd580815fde6974e63ac8e280f1fedb66cd9c59c47566233e4: Status 404 returned error can't find the container with id 726d7a711ebc36dd580815fde6974e63ac8e280f1fedb66cd9c59c47566233e4 Jul 24 13:37:13.831617 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.831579 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:13.831716 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.831663 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:13.831716 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.831704 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:13.953044 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.952964 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" event={"ID":"9b79f972-f9fd-4f53-98a3-4abdc45a68fc","Type":"ContainerStarted","Data":"b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d"} Jul 24 13:37:13.953044 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.953015 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" event={"ID":"9b79f972-f9fd-4f53-98a3-4abdc45a68fc","Type":"ContainerStarted","Data":"726d7a711ebc36dd580815fde6974e63ac8e280f1fedb66cd9c59c47566233e4"} Jul 24 13:37:13.973732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:13.973686 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podStartSLOduration=0.973670063 podStartE2EDuration="973.670063ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:37:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:37:13.972010908 +0000 UTC m=+1660.571256120" watchObservedRunningTime="2026-07-24 13:37:13.973670063 +0000 UTC m=+1660.572915273" Jul 24 13:37:14.687345 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:14.687311 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:14.692351 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:14.692327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:14.961179 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:14.961091 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:14.963768 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:14.963746 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:17.793713 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.793688 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:37:17.905773 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905700 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-socket\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.905773 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905749 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-envoy\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.905982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905786 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-podinfo\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.905982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-credential-socket\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.905982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905818 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-certs\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.905982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905842 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbqx5\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-kube-api-access-vbqx5\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.905982 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905936 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istiod-ca-cert\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.906316 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.905980 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:37:17.906316 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906007 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-data\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.906316 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906067 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-token\") pod \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\" (UID: \"c063dfb7-bc63-42e7-9453-da9a7fc202d9\") " Jul 24 13:37:17.906316 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906140 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:37:17.906316 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906266 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:37:17.906585 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906369 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:17.906585 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906386 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:17.906585 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906396 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:17.906826 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906804 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:37:17.906896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.906866 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-data" (OuterVolumeSpecName: "istio-data") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:37:17.908139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.908115 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:37:17.908231 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.908150 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:37:17.908231 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.908155 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-kube-api-access-vbqx5" (OuterVolumeSpecName: "kube-api-access-vbqx5") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "kube-api-access-vbqx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:37:17.908231 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.908226 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-token" (OuterVolumeSpecName: "istio-token") pod "c063dfb7-bc63-42e7-9453-da9a7fc202d9" (UID: "c063dfb7-bc63-42e7-9453-da9a7fc202d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:37:17.976662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.976634 2562 generic.go:356] "Generic (PLEG): container finished" podID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerID="b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224" exitCode=0 Jul 24 13:37:17.976794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.976678 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" Jul 24 13:37:17.976794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.976710 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" event={"ID":"c063dfb7-bc63-42e7-9453-da9a7fc202d9","Type":"ContainerDied","Data":"b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224"} Jul 24 13:37:17.976794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.976759 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz" event={"ID":"c063dfb7-bc63-42e7-9453-da9a7fc202d9","Type":"ContainerDied","Data":"cfb885ccf2b221eb4ee2475aa64a1808106dc97ded9ae5a1bdd26236f5ae2fcf"} Jul 24 13:37:17.976794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.976780 2562 scope.go:117] "RemoveContainer" containerID="b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224" Jul 24 13:37:17.986284 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.986270 2562 scope.go:117] "RemoveContainer" containerID="d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d" Jul 24 13:37:17.993873 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.993857 2562 scope.go:117] "RemoveContainer" containerID="b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224" Jul 24 13:37:17.994114 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:37:17.994095 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224\": container with ID starting with b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224 not found: ID does not exist" containerID="b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224" Jul 24 13:37:17.994179 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.994127 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224"} err="failed to get container status \"b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224\": rpc error: code = NotFound desc = could not find container \"b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224\": container with ID starting with b76e1732396d8673b3e37b55892a535d968f12af3809366b3fd70e740d6d4224 not found: ID does not exist" Jul 24 13:37:17.994179 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.994147 2562 scope.go:117] "RemoveContainer" containerID="d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d" Jul 24 13:37:17.994377 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:37:17.994362 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d\": container with ID starting with d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d not found: ID does not exist" containerID="d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d" Jul 24 13:37:17.994423 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.994381 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d"} err="failed to get container status \"d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d\": rpc error: code = NotFound desc = could not find container \"d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d\": container with ID starting with d5d4bfc1af17d7536e1bcbc084ce80a2206772d59c6d9333cb201c0541c89f6d not found: ID does not exist" Jul 24 13:37:17.999314 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:17.999293 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz"] Jul 24 13:37:18.003583 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.003539 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-51268fa8-gw-openshift-default-584d47965b-9q8rz"] Jul 24 13:37:18.007736 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.007714 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:18.007822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.007739 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:18.007822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.007752 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:18.007822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.007764 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c063dfb7-bc63-42e7-9453-da9a7fc202d9-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:18.007822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.007777 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c063dfb7-bc63-42e7-9453-da9a7fc202d9-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:18.007822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:18.007792 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbqx5\" (UniqueName: \"kubernetes.io/projected/c063dfb7-bc63-42e7-9453-da9a7fc202d9-kube-api-access-vbqx5\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:37:19.901613 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:19.901577 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" path="/var/lib/kubelet/pods/c063dfb7-bc63-42e7-9453-da9a7fc202d9/volumes" Jul 24 13:37:30.377755 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:30.377663 2562 patch_prober.go:28] interesting pod/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": read tcp 10.133.0.2:38774->10.133.0.46:15021: read: connection reset by peer" start-of-body= Jul 24 13:37:30.378065 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:30.377741 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": read tcp 10.133.0.2:38774->10.133.0.46:15021: read: connection reset by peer" Jul 24 13:37:31.051641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:31.051610 2562 generic.go:356] "Generic (PLEG): container finished" podID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerID="b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d" exitCode=0 Jul 24 13:37:31.051819 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:31.051677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" event={"ID":"9b79f972-f9fd-4f53-98a3-4abdc45a68fc","Type":"ContainerDied","Data":"b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d"} Jul 24 13:37:31.052019 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:31.052007 2562 scope.go:117] "RemoveContainer" containerID="b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d" Jul 24 13:37:31.053061 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:31.053031 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:31.053129 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:31.053107 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:31.053164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:31.053141 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:32.058068 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:32.058033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" event={"ID":"9b79f972-f9fd-4f53-98a3-4abdc45a68fc","Type":"ContainerStarted","Data":"2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d"} Jul 24 13:37:32.686959 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:32.686913 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:32.687188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:32.687074 2562 patch_prober.go:28] interesting pod/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 13:37:32.687188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:32.687121 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 13:37:33.686826 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:33.686789 2562 patch_prober.go:28] interesting pod/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 13:37:33.687411 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:33.687381 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 13:37:34.686937 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:34.686893 2562 patch_prober.go:28] interesting pod/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 13:37:34.687405 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:34.686959 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 13:37:35.687420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:35.687373 2562 patch_prober.go:28] interesting pod/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 24 13:37:35.687931 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:35.687457 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 24 13:37:36.690348 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:36.690321 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:36.690788 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:36.690772 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:36.691356 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:36.691338 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:37:57.594028 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:57.593999 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2"] Jul 24 13:37:57.594501 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:57.594207 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" containerID="cri-o://2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d" gracePeriod=30 Jul 24 13:37:58.413484 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.413445 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm"] Jul 24 13:37:58.413956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.413934 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" Jul 24 13:37:58.413956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.413957 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" Jul 24 13:37:58.414120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.413987 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" Jul 24 13:37:58.414120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.414056 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" Jul 24 13:37:58.414234 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.414167 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" Jul 24 13:37:58.416202 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.416183 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.418895 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.418874 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a35f265b-openshift-default-dockercfg-9n5bv\"" Jul 24 13:37:58.432109 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.432088 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm"] Jul 24 13:37:58.462638 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-token\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.462763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-data\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.462763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462736 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-certs\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.462897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462762 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-socket\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.462897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-podinfo\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.462897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462839 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-credential-socket\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.462897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-envoy\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.463078 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffghl\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-kube-api-access-ffghl\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.463078 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.462952 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istiod-ca-cert\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563398 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-socket\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-podinfo\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-credential-socket\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-envoy\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffghl\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-kube-api-access-ffghl\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istiod-ca-cert\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563642 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-token\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.563698 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563678 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-data\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.564006 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563742 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-certs\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.564006 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.563845 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-socket\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.564127 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.564045 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-certs\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.564166 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.564121 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-data\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.564166 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.564141 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-credential-socket\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.564432 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.564408 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istiod-ca-cert\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.565822 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.565799 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-envoy\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.566078 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.566061 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-podinfo\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.571396 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.571378 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-token\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.571595 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.571575 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffghl\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-kube-api-access-ffghl\") pod \"e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.726679 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.726647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:58.868218 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.868196 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm"] Jul 24 13:37:58.870213 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:37:58.870185 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52c9aa9_ce63_425d_a2d5_1e694c3ad18e.slice/crio-8fd033b0920ba0ef40afa41ef128c4256ca1a0a2fff35aa157594c75b66e779b WatchSource:0}: Error finding container 8fd033b0920ba0ef40afa41ef128c4256ca1a0a2fff35aa157594c75b66e779b: Status 404 returned error can't find the container with id 8fd033b0920ba0ef40afa41ef128c4256ca1a0a2fff35aa157594c75b66e779b Jul 24 13:37:58.872092 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.872061 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:58.872163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.872123 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:58.872163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:58.872150 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:37:59.190874 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:59.190845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" event={"ID":"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e","Type":"ContainerStarted","Data":"e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f"} Jul 24 13:37:59.191046 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:59.190900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" event={"ID":"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e","Type":"ContainerStarted","Data":"8fd033b0920ba0ef40afa41ef128c4256ca1a0a2fff35aa157594c75b66e779b"} Jul 24 13:37:59.211010 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:59.210964 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" podStartSLOduration=1.210949129 podStartE2EDuration="1.210949129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:37:59.209215843 +0000 UTC m=+1705.808461053" watchObservedRunningTime="2026-07-24 13:37:59.210949129 +0000 UTC m=+1705.810194396" Jul 24 13:37:59.727109 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:59.727071 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:37:59.732145 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:37:59.732126 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:00.195910 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:00.195884 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:00.196816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:00.196797 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:02.845166 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.845143 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:38:02.897114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897037 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-credential-socket\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897072 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-certs\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897110 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-socket\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897134 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-podinfo\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897157 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-token\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897424 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897363 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:02.897424 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897382 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-envoy\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897430 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnnpl\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-kube-api-access-pnnpl\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897450 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:02.897540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897465 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:02.897540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897475 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-data\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897540 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897530 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istiod-ca-cert\") pod \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\" (UID: \"9b79f972-f9fd-4f53-98a3-4abdc45a68fc\") " Jul 24 13:38:02.897904 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897853 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:38:02.897979 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897963 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.898034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897984 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.898034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.897997 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.898034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.898009 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.898336 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.898316 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:02.899445 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.899417 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:38:02.899445 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.899431 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:38:02.899795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.899776 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:02.899795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.899784 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-kube-api-access-pnnpl" (OuterVolumeSpecName: "kube-api-access-pnnpl") pod "9b79f972-f9fd-4f53-98a3-4abdc45a68fc" (UID: "9b79f972-f9fd-4f53-98a3-4abdc45a68fc"). InnerVolumeSpecName "kube-api-access-pnnpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:38:02.998395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.998361 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.998395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.998390 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnnpl\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-kube-api-access-pnnpl\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.998395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.998399 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.998637 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.998408 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:02.998637 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:02.998416 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b79f972-f9fd-4f53-98a3-4abdc45a68fc-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:03.215836 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.215802 2562 generic.go:356] "Generic (PLEG): container finished" podID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerID="2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d" exitCode=0 Jul 24 13:38:03.216009 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.215847 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" Jul 24 13:38:03.216009 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.215836 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" event={"ID":"9b79f972-f9fd-4f53-98a3-4abdc45a68fc","Type":"ContainerDied","Data":"2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d"} Jul 24 13:38:03.216009 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.215965 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2" event={"ID":"9b79f972-f9fd-4f53-98a3-4abdc45a68fc","Type":"ContainerDied","Data":"726d7a711ebc36dd580815fde6974e63ac8e280f1fedb66cd9c59c47566233e4"} Jul 24 13:38:03.216009 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.215987 2562 scope.go:117] "RemoveContainer" containerID="2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d" Jul 24 13:38:03.225607 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.225590 2562 scope.go:117] "RemoveContainer" containerID="b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d" Jul 24 13:38:03.234165 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.234150 2562 scope.go:117] "RemoveContainer" containerID="2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d" Jul 24 13:38:03.234394 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:38:03.234376 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d\": container with ID starting with 2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d not found: ID does not exist" containerID="2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d" Jul 24 13:38:03.234442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.234409 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d"} err="failed to get container status \"2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d\": rpc error: code = NotFound desc = could not find container \"2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d\": container with ID starting with 2dcab2b14be688a7251534dfed99624b6bd7b8d6f5e883ab55dad43d92c1ec1d not found: ID does not exist" Jul 24 13:38:03.234442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.234430 2562 scope.go:117] "RemoveContainer" containerID="b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d" Jul 24 13:38:03.234713 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:38:03.234695 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d\": container with ID starting with b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d not found: ID does not exist" containerID="b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d" Jul 24 13:38:03.234782 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.234717 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d"} err="failed to get container status \"b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d\": rpc error: code = NotFound desc = could not find container \"b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d\": container with ID starting with b2b3bf22083b8fb829e537279da7f519ac476e9029ac202aa857dc626521747d not found: ID does not exist" Jul 24 13:38:03.238025 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.238006 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2"] Jul 24 13:38:03.241480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.241460 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ca810d9a-openshift-default-5876b7bdbb-rtvk2"] Jul 24 13:38:03.897460 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:03.897429 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" path="/var/lib/kubelet/pods/9b79f972-f9fd-4f53-98a3-4abdc45a68fc/volumes" Jul 24 13:38:14.273961 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:14.273875 2562 generic.go:356] "Generic (PLEG): container finished" podID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerID="e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f" exitCode=0 Jul 24 13:38:14.273961 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:14.273921 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" event={"ID":"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e","Type":"ContainerDied","Data":"e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f"} Jul 24 13:38:14.274395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:14.274303 2562 scope.go:117] "RemoveContainer" containerID="e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f" Jul 24 13:38:14.275239 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:14.275215 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:38:14.275326 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:14.275274 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:38:14.275326 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:14.275306 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:38:15.196593 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:15.196487 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:15.280766 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:15.280731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" event={"ID":"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e","Type":"ContainerStarted","Data":"2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575"} Jul 24 13:38:15.726859 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:15.726818 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:15.727984 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:15.727959 2562 patch_prober.go:28] interesting pod/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 13:38:15.728108 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:15.728006 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 13:38:16.727883 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:16.727848 2562 patch_prober.go:28] interesting pod/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 13:38:16.728263 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:16.727901 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 13:38:17.727595 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:17.727560 2562 patch_prober.go:28] interesting pod/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 13:38:17.727776 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:17.727608 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 13:38:18.728036 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:18.728001 2562 patch_prober.go:28] interesting pod/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 24 13:38:18.728434 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:18.728065 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 24 13:38:19.730818 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:19.730790 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:19.731207 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:19.731039 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:19.731804 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:19.731790 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:40.731208 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:40.731177 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm"] Jul 24 13:38:40.731753 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:40.731427 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" containerID="cri-o://2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575" gracePeriod=30 Jul 24 13:38:45.974346 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:45.974321 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:46.078747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078653 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffghl\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-kube-api-access-ffghl\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.078747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078715 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-socket\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.078747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078739 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istiod-ca-cert\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078795 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-token\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078822 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-envoy\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078866 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-certs\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078917 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-credential-socket\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078959 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:46.079013 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.078989 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-data\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.079032 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-podinfo\") pod \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\" (UID: \"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e\") " Jul 24 13:38:46.079329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.079167 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:46.079329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.079179 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:46.079329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.079218 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:38:46.080155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.080009 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.080155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.080068 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.080155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.080063 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:46.080398 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.080282 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.080398 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.080322 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.081137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.081113 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:38:46.081265 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.081242 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:38:46.081311 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.081244 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-kube-api-access-ffghl" (OuterVolumeSpecName: "kube-api-access-ffghl") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "kube-api-access-ffghl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:38:46.081349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.081321 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" (UID: "a52c9aa9-ce63-425d-a2d5-1e694c3ad18e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:38:46.180957 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.180924 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.180957 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.180957 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.181155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.180972 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffghl\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-kube-api-access-ffghl\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.181155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.180987 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.181155 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.181000 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:38:46.439180 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.439147 2562 generic.go:356] "Generic (PLEG): container finished" podID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerID="2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575" exitCode=0 Jul 24 13:38:46.439344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.439191 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" Jul 24 13:38:46.439344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.439228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" event={"ID":"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e","Type":"ContainerDied","Data":"2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575"} Jul 24 13:38:46.439344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.439282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm" event={"ID":"a52c9aa9-ce63-425d-a2d5-1e694c3ad18e","Type":"ContainerDied","Data":"8fd033b0920ba0ef40afa41ef128c4256ca1a0a2fff35aa157594c75b66e779b"} Jul 24 13:38:46.439344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.439305 2562 scope.go:117] "RemoveContainer" containerID="2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575" Jul 24 13:38:46.448902 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.448886 2562 scope.go:117] "RemoveContainer" containerID="e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f" Jul 24 13:38:46.456874 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.456859 2562 scope.go:117] "RemoveContainer" containerID="2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575" Jul 24 13:38:46.457110 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:38:46.457090 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575\": container with ID starting with 2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575 not found: ID does not exist" containerID="2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575" Jul 24 13:38:46.457163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.457117 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575"} err="failed to get container status \"2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575\": rpc error: code = NotFound desc = could not find container \"2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575\": container with ID starting with 2f853835269dc94a33d26a348603faaf768cc8116e9adf9bc84f98eba24d5575 not found: ID does not exist" Jul 24 13:38:46.457163 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.457133 2562 scope.go:117] "RemoveContainer" containerID="e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f" Jul 24 13:38:46.457340 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:38:46.457325 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f\": container with ID starting with e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f not found: ID does not exist" containerID="e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f" Jul 24 13:38:46.457380 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.457345 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f"} err="failed to get container status \"e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f\": rpc error: code = NotFound desc = could not find container \"e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f\": container with ID starting with e6dfc291e4eed712acd8b4c2415183329ee853c2b669c77c1177cae590e5121f not found: ID does not exist" Jul 24 13:38:46.461277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.461256 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm"] Jul 24 13:38:46.465061 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:46.465041 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a35f265b-openshift-default-5b75d55544-j4ssm"] Jul 24 13:38:47.896129 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:38:47.896097 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" path="/var/lib/kubelet/pods/a52c9aa9-ce63-425d-a2d5-1e694c3ad18e/volumes" Jul 24 13:40:19.237864 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.237832 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5"] Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238185 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238197 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238208 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238214 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238228 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238233 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238287 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238294 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238299 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c063dfb7-bc63-42e7-9453-da9a7fc202d9" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238304 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" Jul 24 13:40:19.238310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.238310 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b79f972-f9fd-4f53-98a3-4abdc45a68fc" containerName="istio-proxy" Jul 24 13:40:19.241189 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.241169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.243604 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.243582 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-bcb704f0-openshift-default-dockercfg-djqxn\"" Jul 24 13:40:19.259595 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.259573 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5"] Jul 24 13:40:19.387744 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-podinfo\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.387744 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387743 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmf7j\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-kube-api-access-tmf7j\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387766 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-credential-socket\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387783 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-certs\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-envoy\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50abae1-beb9-49a9-9ff7-92422175de0b-istiod-ca-cert\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-data\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.387998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-socket\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.388050 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.388036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-token\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489435 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-podinfo\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489435 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmf7j\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-kube-api-access-tmf7j\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489435 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-credential-socket\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489435 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-certs\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489761 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-envoy\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489761 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50abae1-beb9-49a9-9ff7-92422175de0b-istiod-ca-cert\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489761 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-data\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489885 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489853 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-certs\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.489948 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489900 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-credential-socket\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.490000 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.489940 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-data\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.490051 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.490018 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-socket\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.490099 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.490071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-token\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.490288 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.490266 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-socket\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.490349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.490267 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50abae1-beb9-49a9-9ff7-92422175de0b-istiod-ca-cert\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.491809 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.491789 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-envoy\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.492016 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.491999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-podinfo\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.497230 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.497203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-token\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.497356 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.497340 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmf7j\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-kube-api-access-tmf7j\") pod \"e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.552377 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.552353 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:19.693930 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.693900 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5"] Jul 24 13:40:19.694912 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:40:19.694889 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50abae1_beb9_49a9_9ff7_92422175de0b.slice/crio-6de330055477403a44d0f4069d70528dadbfa9527c0a60398df0d03c37044fb7 WatchSource:0}: Error finding container 6de330055477403a44d0f4069d70528dadbfa9527c0a60398df0d03c37044fb7: Status 404 returned error can't find the container with id 6de330055477403a44d0f4069d70528dadbfa9527c0a60398df0d03c37044fb7 Jul 24 13:40:19.696937 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.696909 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:40:19.697031 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.696967 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:40:19.697031 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.697008 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:40:19.905397 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.905367 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" event={"ID":"f50abae1-beb9-49a9-9ff7-92422175de0b","Type":"ContainerStarted","Data":"adf1049d985c689fddc941cdbb39c63d2e9032718b518d1211e114d9c618500c"} Jul 24 13:40:19.905537 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.905408 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" event={"ID":"f50abae1-beb9-49a9-9ff7-92422175de0b","Type":"ContainerStarted","Data":"6de330055477403a44d0f4069d70528dadbfa9527c0a60398df0d03c37044fb7"} Jul 24 13:40:19.925347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:19.925288 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" podStartSLOduration=0.925268269 podStartE2EDuration="925.268269ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:40:19.924404553 +0000 UTC m=+1846.523649765" watchObservedRunningTime="2026-07-24 13:40:19.925268269 +0000 UTC m=+1846.524513480" Jul 24 13:40:20.553015 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:20.552983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:20.558097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:20.558072 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:20.911197 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:20.911123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:20.912051 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:20.912028 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:32.973293 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:32.973263 2562 generic.go:356] "Generic (PLEG): container finished" podID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerID="adf1049d985c689fddc941cdbb39c63d2e9032718b518d1211e114d9c618500c" exitCode=0 Jul 24 13:40:32.973712 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:32.973297 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" event={"ID":"f50abae1-beb9-49a9-9ff7-92422175de0b","Type":"ContainerDied","Data":"adf1049d985c689fddc941cdbb39c63d2e9032718b518d1211e114d9c618500c"} Jul 24 13:40:32.973712 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:32.973675 2562 scope.go:117] "RemoveContainer" containerID="adf1049d985c689fddc941cdbb39c63d2e9032718b518d1211e114d9c618500c" Jul 24 13:40:32.974687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:32.974658 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:40:32.974766 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:32.974741 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:40:32.974808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:32.974791 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:40:33.980299 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:33.980264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" event={"ID":"f50abae1-beb9-49a9-9ff7-92422175de0b","Type":"ContainerStarted","Data":"857950f2a9e4903cc92580e73b469e48e268b5bfefbdd61c0e6a4f59689ea82f"} Jul 24 13:40:34.552914 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:34.552881 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:34.553981 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:34.553950 2562 patch_prober.go:28] interesting pod/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 13:40:34.554115 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:34.553997 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 13:40:35.552736 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:35.552702 2562 patch_prober.go:28] interesting pod/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 13:40:35.553114 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:35.552759 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 13:40:35.911311 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:35.911221 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:36.553301 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:36.553264 2562 patch_prober.go:28] interesting pod/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 13:40:36.553726 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:36.553319 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 13:40:37.553441 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:37.553401 2562 patch_prober.go:28] interesting pod/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 24 13:40:37.553854 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:37.553457 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 24 13:40:38.556838 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:38.556809 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:38.557979 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:38.557962 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:40:55.938572 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:55.938482 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5"] Jul 24 13:40:55.939338 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:40:55.938733 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" containerID="cri-o://857950f2a9e4903cc92580e73b469e48e268b5bfefbdd61c0e6a4f59689ea82f" gracePeriod=30 Jul 24 13:41:01.132767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.132731 2562 generic.go:356] "Generic (PLEG): container finished" podID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerID="857950f2a9e4903cc92580e73b469e48e268b5bfefbdd61c0e6a4f59689ea82f" exitCode=0 Jul 24 13:41:01.133136 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.132795 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" event={"ID":"f50abae1-beb9-49a9-9ff7-92422175de0b","Type":"ContainerDied","Data":"857950f2a9e4903cc92580e73b469e48e268b5bfefbdd61c0e6a4f59689ea82f"} Jul 24 13:41:01.133136 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.132860 2562 scope.go:117] "RemoveContainer" containerID="adf1049d985c689fddc941cdbb39c63d2e9032718b518d1211e114d9c618500c" Jul 24 13:41:01.188582 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.188557 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:41:01.257290 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257217 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-token\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257290 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257246 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50abae1-beb9-49a9-9ff7-92422175de0b-istiod-ca-cert\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257290 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257268 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-credential-socket\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257320 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-podinfo\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257349 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-certs\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257379 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-socket\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257723 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257582 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-data\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257723 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257665 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmf7j\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-kube-api-access-tmf7j\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257723 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257673 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:01.257723 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257694 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-envoy\") pod \"f50abae1-beb9-49a9-9ff7-92422175de0b\" (UID: \"f50abae1-beb9-49a9-9ff7-92422175de0b\") " Jul 24 13:41:01.257723 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257702 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:01.257723 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257712 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:01.257951 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.257753 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50abae1-beb9-49a9-9ff7-92422175de0b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:41:01.258017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.258001 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50abae1-beb9-49a9-9ff7-92422175de0b-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.258065 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.258023 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.258065 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.258037 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.258065 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.258049 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.258485 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.258457 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-data" (OuterVolumeSpecName: "istio-data") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:01.259680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.259661 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:41:01.259803 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.259785 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-token" (OuterVolumeSpecName: "istio-token") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:41:01.259986 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.259960 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:41:01.259986 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.259975 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-kube-api-access-tmf7j" (OuterVolumeSpecName: "kube-api-access-tmf7j") pod "f50abae1-beb9-49a9-9ff7-92422175de0b" (UID: "f50abae1-beb9-49a9-9ff7-92422175de0b"). InnerVolumeSpecName "kube-api-access-tmf7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:41:01.359214 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.359185 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmf7j\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-kube-api-access-tmf7j\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.359214 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.359208 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.359214 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.359220 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.359417 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.359228 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:01.359417 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:01.359235 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50abae1-beb9-49a9-9ff7-92422175de0b-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:41:02.140154 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:02.140125 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" Jul 24 13:41:02.140616 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:02.140151 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5" event={"ID":"f50abae1-beb9-49a9-9ff7-92422175de0b","Type":"ContainerDied","Data":"6de330055477403a44d0f4069d70528dadbfa9527c0a60398df0d03c37044fb7"} Jul 24 13:41:02.140616 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:02.140207 2562 scope.go:117] "RemoveContainer" containerID="857950f2a9e4903cc92580e73b469e48e268b5bfefbdd61c0e6a4f59689ea82f" Jul 24 13:41:02.157791 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:02.157739 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5"] Jul 24 13:41:02.161562 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:02.161528 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-bcb704f0-openshift-default-7bb5f94c68-chjk5"] Jul 24 13:41:03.897312 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:03.897278 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" path="/var/lib/kubelet/pods/f50abae1-beb9-49a9-9ff7-92422175de0b/volumes" Jul 24 13:41:16.232488 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232449 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct"] Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232854 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232869 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232881 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232886 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52c9aa9-ce63-425d-a2d5-1e694c3ad18e" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232895 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232900 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232986 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" Jul 24 13:41:16.233038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.232999 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50abae1-beb9-49a9-9ff7-92422175de0b" containerName="istio-proxy" Jul 24 13:41:16.235878 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.235859 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.238421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.238402 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2cdea471-openshift-default-dockercfg-w482h\"" Jul 24 13:41:16.249768 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.249747 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct"] Jul 24 13:41:16.290320 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290295 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-data\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30f0ea4e-c75c-4ea5-b228-c343208df661-istiod-ca-cert\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-credential-socket\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290560 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-socket\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290560 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-token\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-certs\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290617 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc4mh\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-kube-api-access-lc4mh\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-envoy\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.290805 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.290711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-podinfo\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391620 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-token\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-certs\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391655 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc4mh\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-kube-api-access-lc4mh\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-envoy\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391716 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-podinfo\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-data\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30f0ea4e-c75c-4ea5-b228-c343208df661-istiod-ca-cert\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-credential-socket\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.391827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.391830 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-socket\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.392258 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.392108 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-certs\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.392258 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.392161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-data\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.392258 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.392229 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-socket\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.392418 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.392288 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-credential-socket\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.392522 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.392500 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30f0ea4e-c75c-4ea5-b228-c343208df661-istiod-ca-cert\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.393986 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.393966 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-envoy\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.394300 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.394282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-podinfo\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.405777 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.405753 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-token\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.405872 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.405813 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc4mh\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-kube-api-access-lc4mh\") pod \"e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.546111 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.546036 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:16.678298 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.678272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct"] Jul 24 13:41:16.679677 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:41:16.679655 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30f0ea4e_c75c_4ea5_b228_c343208df661.slice/crio-0974db613efb1017cae1eb232337762b594efbc61913d6f2c57e51872d49fa91 WatchSource:0}: Error finding container 0974db613efb1017cae1eb232337762b594efbc61913d6f2c57e51872d49fa91: Status 404 returned error can't find the container with id 0974db613efb1017cae1eb232337762b594efbc61913d6f2c57e51872d49fa91 Jul 24 13:41:16.681747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.681720 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:41:16.681791 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.681779 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:41:16.681825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:16.681807 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:41:17.229919 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:17.229880 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" event={"ID":"30f0ea4e-c75c-4ea5-b228-c343208df661","Type":"ContainerStarted","Data":"d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358"} Jul 24 13:41:17.230093 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:17.229964 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" event={"ID":"30f0ea4e-c75c-4ea5-b228-c343208df661","Type":"ContainerStarted","Data":"0974db613efb1017cae1eb232337762b594efbc61913d6f2c57e51872d49fa91"} Jul 24 13:41:17.251199 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:17.251155 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podStartSLOduration=1.2511417599999999 podStartE2EDuration="1.25114176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:41:17.248861443 +0000 UTC m=+1903.848106655" watchObservedRunningTime="2026-07-24 13:41:17.25114176 +0000 UTC m=+1903.850386972" Jul 24 13:41:17.546761 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:17.546681 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:17.551439 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:17.551415 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:18.235120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:18.235091 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:18.235911 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:18.235889 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:29.113571 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.112320 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4"] Jul 24 13:41:29.117188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.116567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.120128 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.120100 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2cdea471\"/\"kube-root-ca.crt\"" Jul 24 13:41:29.120943 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.120872 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2cdea471\"/\"openshift-service-ca.crt\"" Jul 24 13:41:29.121119 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.120992 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2cdea471\"/\"default-dockercfg-25mfn\"" Jul 24 13:41:29.121457 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.121440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2cdea471\"/\"e2e-lifecycle-model-2cdea471-kserve-self-signed-certs\"" Jul 24 13:41:29.124819 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.124795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4"] Jul 24 13:41:29.217794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.217767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-home\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.217929 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.217796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tmp-dir\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.217929 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.217835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-dshm\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.218027 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.217930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-model-cache\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.218027 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.217970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q9g2\" (UniqueName: \"kubernetes.io/projected/d3fcaee8-a999-4916-8794-a43fdcdf40f9-kube-api-access-8q9g2\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.218027 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.218000 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tls-certs\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319074 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-dshm\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319074 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-model-cache\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q9g2\" (UniqueName: \"kubernetes.io/projected/d3fcaee8-a999-4916-8794-a43fdcdf40f9-kube-api-access-8q9g2\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tls-certs\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-home\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319344 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319336 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tmp-dir\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319591 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319523 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-model-cache\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319733 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-home\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.319807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.319713 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tmp-dir\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.321473 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.321444 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-dshm\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.321751 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.321731 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tls-certs\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.327665 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.327646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q9g2\" (UniqueName: \"kubernetes.io/projected/d3fcaee8-a999-4916-8794-a43fdcdf40f9-kube-api-access-8q9g2\") pod \"e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.429866 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.429832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:29.610657 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:29.610620 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4"] Jul 24 13:41:29.614977 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:41:29.614948 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3fcaee8_a999_4916_8794_a43fdcdf40f9.slice/crio-500530265fd964ca3aa921f1020ba25959c18954110fbfb0dd8ec0e30eecaa50 WatchSource:0}: Error finding container 500530265fd964ca3aa921f1020ba25959c18954110fbfb0dd8ec0e30eecaa50: Status 404 returned error can't find the container with id 500530265fd964ca3aa921f1020ba25959c18954110fbfb0dd8ec0e30eecaa50 Jul 24 13:41:30.309972 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:30.309933 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" event={"ID":"d3fcaee8-a999-4916-8794-a43fdcdf40f9","Type":"ContainerStarted","Data":"92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f"} Jul 24 13:41:30.310425 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:30.309987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" event={"ID":"d3fcaee8-a999-4916-8794-a43fdcdf40f9","Type":"ContainerStarted","Data":"500530265fd964ca3aa921f1020ba25959c18954110fbfb0dd8ec0e30eecaa50"} Jul 24 13:41:30.327271 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:30.327227 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" podStartSLOduration=1.327211389 podStartE2EDuration="1.327211389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:41:30.325791848 +0000 UTC m=+1916.925037059" watchObservedRunningTime="2026-07-24 13:41:30.327211389 +0000 UTC m=+1916.926456600" Jul 24 13:41:33.488750 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:33.488307 2562 patch_prober.go:28] interesting pod/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:37996->10.133.0.49:15021: read: connection reset by peer" start-of-body= Jul 24 13:41:33.488750 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:33.488382 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": read tcp 10.133.0.2:37996->10.133.0.49:15021: read: connection reset by peer" Jul 24 13:41:34.333264 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:34.333182 2562 generic.go:356] "Generic (PLEG): container finished" podID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerID="d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358" exitCode=0 Jul 24 13:41:34.333264 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:34.333219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" event={"ID":"30f0ea4e-c75c-4ea5-b228-c343208df661","Type":"ContainerDied","Data":"d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358"} Jul 24 13:41:34.333629 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:34.333615 2562 scope.go:117] "RemoveContainer" containerID="d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358" Jul 24 13:41:34.334607 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:34.334577 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:41:34.334692 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:34.334652 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:41:34.334692 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:34.334683 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:41:35.340604 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:35.340574 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" event={"ID":"30f0ea4e-c75c-4ea5-b228-c343208df661","Type":"ContainerStarted","Data":"bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e"} Jul 24 13:41:35.546262 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:35.546225 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:35.546443 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:35.546421 2562 patch_prober.go:28] interesting pod/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 13:41:35.546490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:35.546468 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 13:41:36.547403 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:36.547366 2562 patch_prober.go:28] interesting pod/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 13:41:36.547806 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:36.547434 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 13:41:37.546904 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:37.546866 2562 patch_prober.go:28] interesting pod/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 13:41:37.547112 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:37.546937 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 13:41:38.547107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:38.547070 2562 patch_prober.go:28] interesting pod/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 24 13:41:38.547501 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:38.547121 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 24 13:41:39.430411 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:39.430376 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:39.430608 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:39.430451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:39.442820 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:39.442802 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:41:39.550533 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:39.550507 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:39.550905 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:39.550760 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:39.551516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:39.551499 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:41:40.375608 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:41:40.375581 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:42:27.064656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:27.064576 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4"] Jul 24 13:42:42.083670 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.083608 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" podUID="d3fcaee8-a999-4916-8794-a43fdcdf40f9" containerName="main" containerID="cri-o://92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f" gracePeriod=45 Jul 24 13:42:42.329923 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.329897 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:42:42.389226 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389153 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-home\") pod \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " Jul 24 13:42:42.389226 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389183 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tmp-dir\") pod \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " Jul 24 13:42:42.389226 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389202 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-dshm\") pod \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " Jul 24 13:42:42.389482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389230 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tls-certs\") pod \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " Jul 24 13:42:42.389482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8q9g2\" (UniqueName: \"kubernetes.io/projected/d3fcaee8-a999-4916-8794-a43fdcdf40f9-kube-api-access-8q9g2\") pod \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " Jul 24 13:42:42.389482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389361 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-model-cache\") pod \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\" (UID: \"d3fcaee8-a999-4916-8794-a43fdcdf40f9\") " Jul 24 13:42:42.389680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389436 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3fcaee8-a999-4916-8794-a43fdcdf40f9" (UID: "d3fcaee8-a999-4916-8794-a43fdcdf40f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:42:42.389680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389484 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-home" (OuterVolumeSpecName: "home") pod "d3fcaee8-a999-4916-8794-a43fdcdf40f9" (UID: "d3fcaee8-a999-4916-8794-a43fdcdf40f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:42:42.389793 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389686 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3fcaee8-a999-4916-8794-a43fdcdf40f9" (UID: "d3fcaee8-a999-4916-8794-a43fdcdf40f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:42:42.389793 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389764 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-home\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:42:42.389793 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.389788 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tmp-dir\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:42:42.391412 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.391379 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-dshm" (OuterVolumeSpecName: "dshm") pod "d3fcaee8-a999-4916-8794-a43fdcdf40f9" (UID: "d3fcaee8-a999-4916-8794-a43fdcdf40f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:42:42.391569 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.391438 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3fcaee8-a999-4916-8794-a43fdcdf40f9" (UID: "d3fcaee8-a999-4916-8794-a43fdcdf40f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:42:42.391569 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.391482 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3fcaee8-a999-4916-8794-a43fdcdf40f9-kube-api-access-8q9g2" (OuterVolumeSpecName: "kube-api-access-8q9g2") pod "d3fcaee8-a999-4916-8794-a43fdcdf40f9" (UID: "d3fcaee8-a999-4916-8794-a43fdcdf40f9"). InnerVolumeSpecName "kube-api-access-8q9g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:42:42.490741 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.490712 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-model-cache\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:42:42.490741 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.490737 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3fcaee8-a999-4916-8794-a43fdcdf40f9-dshm\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:42:42.490741 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.490745 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3fcaee8-a999-4916-8794-a43fdcdf40f9-tls-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:42:42.490947 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.490754 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8q9g2\" (UniqueName: \"kubernetes.io/projected/d3fcaee8-a999-4916-8794-a43fdcdf40f9-kube-api-access-8q9g2\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:42:42.672200 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.672121 2562 generic.go:356] "Generic (PLEG): container finished" podID="d3fcaee8-a999-4916-8794-a43fdcdf40f9" containerID="92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f" exitCode=0 Jul 24 13:42:42.672200 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.672161 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" Jul 24 13:42:42.672200 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.672181 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" event={"ID":"d3fcaee8-a999-4916-8794-a43fdcdf40f9","Type":"ContainerDied","Data":"92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f"} Jul 24 13:42:42.672435 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.672210 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4" event={"ID":"d3fcaee8-a999-4916-8794-a43fdcdf40f9","Type":"ContainerDied","Data":"500530265fd964ca3aa921f1020ba25959c18954110fbfb0dd8ec0e30eecaa50"} Jul 24 13:42:42.672435 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.672226 2562 scope.go:117] "RemoveContainer" containerID="92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f" Jul 24 13:42:42.681348 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.681332 2562 scope.go:117] "RemoveContainer" containerID="92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f" Jul 24 13:42:42.681659 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:42:42.681641 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f\": container with ID starting with 92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f not found: ID does not exist" containerID="92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f" Jul 24 13:42:42.681731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.681670 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f"} err="failed to get container status \"92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f\": rpc error: code = NotFound desc = could not find container \"92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f\": container with ID starting with 92e221dc76a800a54c363cae9befff692e713658691099ef34aac35f8fe3be9f not found: ID does not exist" Jul 24 13:42:42.693206 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.693181 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4"] Jul 24 13:42:42.697024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:42.697001 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2cdea471/e2e-lifecycle-model-2cdea471-kserve-58df89f7f-9spn4"] Jul 24 13:42:43.896141 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:42:43.896108 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3fcaee8-a999-4916-8794-a43fdcdf40f9" path="/var/lib/kubelet/pods/d3fcaee8-a999-4916-8794-a43fdcdf40f9/volumes" Jul 24 13:43:33.316008 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:33.315972 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct"] Jul 24 13:43:33.316439 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:33.316188 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" containerID="cri-o://bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e" gracePeriod=30 Jul 24 13:43:38.570127 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.570100 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:43:38.669748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669674 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-credential-socket\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.669748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669706 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc4mh\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-kube-api-access-lc4mh\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.669748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669727 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-data\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669780 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-podinfo\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669802 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30f0ea4e-c75c-4ea5-b228-c343208df661-istiod-ca-cert\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669832 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-socket\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669865 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-token\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669897 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-envoy\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669936 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-certs\") pod \"30f0ea4e-c75c-4ea5-b228-c343208df661\" (UID: \"30f0ea4e-c75c-4ea5-b228-c343208df661\") " Jul 24 13:43:38.670023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.669946 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:43:38.670357 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.670218 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.670357 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.670248 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:43:38.670577 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.670528 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:43:38.670665 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.670635 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30f0ea4e-c75c-4ea5-b228-c343208df661-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:43:38.671025 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.670999 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-data" (OuterVolumeSpecName: "istio-data") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:43:38.672056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.672028 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:43:38.672150 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.672114 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-kube-api-access-lc4mh" (OuterVolumeSpecName: "kube-api-access-lc4mh") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "kube-api-access-lc4mh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:43:38.672150 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.672126 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-token" (OuterVolumeSpecName: "istio-token") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:43:38.672244 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.672222 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30f0ea4e-c75c-4ea5-b228-c343208df661" (UID: "30f0ea4e-c75c-4ea5-b228-c343208df661"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:43:38.770867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770842 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.770867 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770864 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30f0ea4e-c75c-4ea5-b228-c343208df661-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.771021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770874 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.771021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770883 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.771021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770892 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.771021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770899 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.771021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770907 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc4mh\" (UniqueName: \"kubernetes.io/projected/30f0ea4e-c75c-4ea5-b228-c343208df661-kube-api-access-lc4mh\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.771021 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.770916 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30f0ea4e-c75c-4ea5-b228-c343208df661-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:43:38.960170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.960143 2562 generic.go:356] "Generic (PLEG): container finished" podID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerID="bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e" exitCode=0 Jul 24 13:43:38.960312 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.960191 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" Jul 24 13:43:38.960312 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.960228 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" event={"ID":"30f0ea4e-c75c-4ea5-b228-c343208df661","Type":"ContainerDied","Data":"bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e"} Jul 24 13:43:38.960312 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.960285 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct" event={"ID":"30f0ea4e-c75c-4ea5-b228-c343208df661","Type":"ContainerDied","Data":"0974db613efb1017cae1eb232337762b594efbc61913d6f2c57e51872d49fa91"} Jul 24 13:43:38.960431 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.960315 2562 scope.go:117] "RemoveContainer" containerID="bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e" Jul 24 13:43:38.969504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.969488 2562 scope.go:117] "RemoveContainer" containerID="d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358" Jul 24 13:43:38.977183 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.977168 2562 scope.go:117] "RemoveContainer" containerID="bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e" Jul 24 13:43:38.977429 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:43:38.977412 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e\": container with ID starting with bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e not found: ID does not exist" containerID="bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e" Jul 24 13:43:38.977466 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.977440 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e"} err="failed to get container status \"bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e\": rpc error: code = NotFound desc = could not find container \"bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e\": container with ID starting with bb436ed3bd88f5f322ac13ec98fc1d1368e84bb420dc82e9494baa28698a789e not found: ID does not exist" Jul 24 13:43:38.977466 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.977457 2562 scope.go:117] "RemoveContainer" containerID="d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358" Jul 24 13:43:38.977706 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:43:38.977687 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358\": container with ID starting with d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358 not found: ID does not exist" containerID="d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358" Jul 24 13:43:38.977748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.977713 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358"} err="failed to get container status \"d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358\": rpc error: code = NotFound desc = could not find container \"d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358\": container with ID starting with d80f5aa9d219673692dd841682d91e79d0ae84aa8d6fcc16b94aad13d9873358 not found: ID does not exist" Jul 24 13:43:38.982018 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.981999 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct"] Jul 24 13:43:38.989441 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:38.988530 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2cdea471-openshift-default-7dfd755dcb-n2zct"] Jul 24 13:43:39.896043 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:43:39.896007 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" path="/var/lib/kubelet/pods/30f0ea4e-c75c-4ea5-b228-c343208df661/volumes" Jul 24 13:44:31.385275 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385237 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg"] Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385609 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385621 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385632 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385638 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385647 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3fcaee8-a999-4916-8794-a43fdcdf40f9" containerName="main" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385653 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3fcaee8-a999-4916-8794-a43fdcdf40f9" containerName="main" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385706 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3fcaee8-a999-4916-8794-a43fdcdf40f9" containerName="main" Jul 24 13:44:31.385715 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385716 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" Jul 24 13:44:31.385990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.385724 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="30f0ea4e-c75c-4ea5-b228-c343208df661" containerName="istio-proxy" Jul 24 13:44:31.388868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.388848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.391228 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.391205 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-50b298-openshift-default-dockercfg-dxkkd\"" Jul 24 13:44:31.407129 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.407107 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg"] Jul 24 13:44:31.436904 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.436881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-data\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437022 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.436926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istiod-ca-cert\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437022 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.436984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-certs\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.437039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-socket\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.437077 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-envoy\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.437104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-podinfo\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.437129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-credential-socket\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437352 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.437188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4n5f\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-kube-api-access-n4n5f\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.437352 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.437247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-token\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538632 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538594 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-socket\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538643 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-envoy\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-podinfo\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538816 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538793 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-credential-socket\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4n5f\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-kube-api-access-n4n5f\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538890 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-token\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-data\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istiod-ca-cert\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.538990 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-certs\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.539272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.538986 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-socket\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.539272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.539164 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-credential-socket\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.539391 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.539308 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-certs\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.539391 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.539319 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-data\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.539788 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.539766 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istiod-ca-cert\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.540975 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.540952 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-envoy\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.541245 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.541227 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-podinfo\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.547184 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.547165 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4n5f\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-kube-api-access-n4n5f\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.547263 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.547251 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-token\") pod \"e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.699130 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.699102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:31.833696 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.833671 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg"] Jul 24 13:44:31.835804 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:44:31.835779 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda81dfb27_ada9_4b00_b2c2_fa4656d2ab7e.slice/crio-e31543cc864db346cf8cd2efdbd9a7cda9db99dc5463588abcfa7af875332ae3 WatchSource:0}: Error finding container e31543cc864db346cf8cd2efdbd9a7cda9db99dc5463588abcfa7af875332ae3: Status 404 returned error can't find the container with id e31543cc864db346cf8cd2efdbd9a7cda9db99dc5463588abcfa7af875332ae3 Jul 24 13:44:31.839101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.838693 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:31.839101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.838774 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:31.839101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:31.838821 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:32.221273 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:32.221240 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" event={"ID":"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e","Type":"ContainerStarted","Data":"f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb"} Jul 24 13:44:32.221437 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:32.221294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" event={"ID":"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e","Type":"ContainerStarted","Data":"e31543cc864db346cf8cd2efdbd9a7cda9db99dc5463588abcfa7af875332ae3"} Jul 24 13:44:32.242374 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:32.242328 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podStartSLOduration=1.242312167 podStartE2EDuration="1.242312167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:32.240783475 +0000 UTC m=+2098.840028686" watchObservedRunningTime="2026-07-24 13:44:32.242312167 +0000 UTC m=+2098.841557377" Jul 24 13:44:32.699281 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:32.699244 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:32.704331 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:32.704310 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:33.233134 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:33.233097 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:33.234063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:33.234043 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:39.377398 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.377363 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7"] Jul 24 13:44:39.380956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.380935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.383434 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.383415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7918ee-openshift-default-dockercfg-45npj\"" Jul 24 13:44:39.391898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.391879 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7"] Jul 24 13:44:39.516567 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516520 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-socket\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516759 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516601 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-certs\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516759 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-credential-socket\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516759 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2687f56-9671-44be-ba87-4d34139398f1-istio-podinfo\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516759 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2687f56-9671-44be-ba87-4d34139398f1-istiod-ca-cert\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516932 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-data\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516932 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvszh\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-kube-api-access-gvszh\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.516932 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-envoy\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.517043 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.516964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-istio-token\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-data\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618223 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvszh\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-kube-api-access-gvszh\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-envoy\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-istio-token\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-socket\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618322 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-certs\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-credential-socket\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2687f56-9671-44be-ba87-4d34139398f1-istio-podinfo\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2687f56-9671-44be-ba87-4d34139398f1-istiod-ca-cert\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618811 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618652 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-data\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618924 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618901 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-certs\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618986 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618955 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-socket\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.618986 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.618974 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-credential-socket\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.619225 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.619207 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2687f56-9671-44be-ba87-4d34139398f1-istiod-ca-cert\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.620607 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.620583 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-envoy\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.620975 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.620958 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2687f56-9671-44be-ba87-4d34139398f1-istio-podinfo\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.625959 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.625939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-istio-token\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.626107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.626091 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvszh\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-kube-api-access-gvszh\") pod \"e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.691749 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.691726 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:39.831157 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.831133 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7"] Jul 24 13:44:39.833413 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:44:39.833388 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2687f56_9671_44be_ba87_4d34139398f1.slice/crio-a7ef54ee2803453e7fb6eb1473d41fa516f178ee22744308f5914de159a44b75 WatchSource:0}: Error finding container a7ef54ee2803453e7fb6eb1473d41fa516f178ee22744308f5914de159a44b75: Status 404 returned error can't find the container with id a7ef54ee2803453e7fb6eb1473d41fa516f178ee22744308f5914de159a44b75 Jul 24 13:44:39.835529 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.835501 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:39.835635 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.835578 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:39.835635 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:39.835607 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:40.272438 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:40.272406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" event={"ID":"e2687f56-9671-44be-ba87-4d34139398f1","Type":"ContainerStarted","Data":"62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f"} Jul 24 13:44:40.272641 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:40.272461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" event={"ID":"e2687f56-9671-44be-ba87-4d34139398f1","Type":"ContainerStarted","Data":"a7ef54ee2803453e7fb6eb1473d41fa516f178ee22744308f5914de159a44b75"} Jul 24 13:44:40.293012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:40.292962 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podStartSLOduration=1.29294642 podStartE2EDuration="1.29294642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:44:40.291188309 +0000 UTC m=+2106.890433550" watchObservedRunningTime="2026-07-24 13:44:40.29294642 +0000 UTC m=+2106.892191631" Jul 24 13:44:40.692606 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:40.692575 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:40.697199 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:40.697176 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:41.277590 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:41.277560 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:41.278482 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:41.278465 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:49.273707 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:49.273664 2562 patch_prober.go:28] interesting pod/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 13:44:49.274126 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:49.273732 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 13:44:50.336751 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:50.336714 2562 generic.go:356] "Generic (PLEG): container finished" podID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerID="f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb" exitCode=0 Jul 24 13:44:50.337117 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:50.336764 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" event={"ID":"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e","Type":"ContainerDied","Data":"f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb"} Jul 24 13:44:50.337158 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:50.337126 2562 scope.go:117] "RemoveContainer" containerID="f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb" Jul 24 13:44:50.338063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:50.338039 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:50.338120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:50.338101 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:50.338161 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:50.338133 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:51.346102 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:51.346065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" event={"ID":"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e","Type":"ContainerStarted","Data":"f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6"} Jul 24 13:44:51.700364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:51.699600 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:51.700364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:51.700024 2562 patch_prober.go:28] interesting pod/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 13:44:51.700364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:51.700086 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 13:44:52.700821 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:52.700324 2562 patch_prober.go:28] interesting pod/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 13:44:52.700821 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:52.700390 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 13:44:53.699507 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:53.699471 2562 patch_prober.go:28] interesting pod/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 13:44:53.699829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:53.699800 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 13:44:54.699803 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:54.699759 2562 patch_prober.go:28] interesting pod/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 24 13:44:54.700298 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:54.699827 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 24 13:44:55.703836 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:55.703802 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:55.704246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:55.704158 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:55.704793 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:55.704775 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:44:57.318687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:57.318652 2562 patch_prober.go:28] interesting pod/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 13:44:57.319130 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:57.318721 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 13:44:58.383386 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:58.383354 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2687f56-9671-44be-ba87-4d34139398f1" containerID="62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f" exitCode=0 Jul 24 13:44:58.383819 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:58.383427 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" event={"ID":"e2687f56-9671-44be-ba87-4d34139398f1","Type":"ContainerDied","Data":"62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f"} Jul 24 13:44:58.383862 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:58.383834 2562 scope.go:117] "RemoveContainer" containerID="62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f" Jul 24 13:44:58.384874 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:58.384848 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:58.384947 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:58.384932 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:58.384989 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:58.384969 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:44:59.394971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:59.394928 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" event={"ID":"e2687f56-9671-44be-ba87-4d34139398f1","Type":"ContainerStarted","Data":"c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c"} Jul 24 13:44:59.692521 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:59.692481 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:44:59.692756 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:59.692652 2562 patch_prober.go:28] interesting pod/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 13:44:59.692756 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:44:59.692699 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 13:45:00.693968 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:00.693931 2562 patch_prober.go:28] interesting pod/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 13:45:00.694602 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:00.694569 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 13:45:01.692268 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:01.692228 2562 patch_prober.go:28] interesting pod/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 13:45:01.692586 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:01.692530 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 13:45:02.692963 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:02.692921 2562 patch_prober.go:28] interesting pod/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 13:45:02.693620 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:02.693590 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 13:45:03.696462 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:03.696427 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:45:03.696901 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:03.696804 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:45:03.697648 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:45:03.697627 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:46:34.382488 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:34.382443 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg"] Jul 24 13:46:34.383023 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:34.382736 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" containerID="cri-o://f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6" gracePeriod=30 Jul 24 13:46:39.636728 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.636704 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:46:39.746731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746702 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-credential-socket\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746743 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-envoy\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746791 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-data\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746807 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istiod-ca-cert\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746840 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4n5f\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-kube-api-access-n4n5f\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746858 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-token\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746880 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-socket\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.746913 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746914 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-certs\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.747263 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746951 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-podinfo\") pod \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\" (UID: \"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e\") " Jul 24 13:46:39.747263 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.746983 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:46:39.747263 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.747250 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.747411 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.747319 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:46:39.747411 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.747342 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:46:39.747598 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.747580 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:46:39.747808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.747786 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:46:39.749126 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.749093 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:46:39.749215 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.749193 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-kube-api-access-n4n5f" (OuterVolumeSpecName: "kube-api-access-n4n5f") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "kube-api-access-n4n5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:46:39.749428 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.749403 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:46:39.749522 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.749458 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" (UID: "a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:46:39.847835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847809 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.847835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847831 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.847835 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847839 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.848041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847848 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4n5f\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-kube-api-access-n4n5f\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.848041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847856 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.848041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847866 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.848041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847873 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.848041 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.847881 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:46:39.920371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.920342 2562 generic.go:356] "Generic (PLEG): container finished" podID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerID="f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6" exitCode=0 Jul 24 13:46:39.920504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.920390 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" Jul 24 13:46:39.920504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.920426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" event={"ID":"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e","Type":"ContainerDied","Data":"f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6"} Jul 24 13:46:39.920504 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.920479 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg" event={"ID":"a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e","Type":"ContainerDied","Data":"e31543cc864db346cf8cd2efdbd9a7cda9db99dc5463588abcfa7af875332ae3"} Jul 24 13:46:39.920672 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.920510 2562 scope.go:117] "RemoveContainer" containerID="f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6" Jul 24 13:46:39.929479 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.929461 2562 scope.go:117] "RemoveContainer" containerID="f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb" Jul 24 13:46:39.938032 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.938011 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg"] Jul 24 13:46:39.938359 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.938341 2562 scope.go:117] "RemoveContainer" containerID="f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6" Jul 24 13:46:39.938611 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:46:39.938591 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6\": container with ID starting with f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6 not found: ID does not exist" containerID="f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6" Jul 24 13:46:39.938674 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.938620 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6"} err="failed to get container status \"f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6\": rpc error: code = NotFound desc = could not find container \"f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6\": container with ID starting with f1bcbec0734a9b4c6ab9d53ca6cba90c993d01da94e8a4e3d7155e69b8e8f5b6 not found: ID does not exist" Jul 24 13:46:39.938674 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.938635 2562 scope.go:117] "RemoveContainer" containerID="f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb" Jul 24 13:46:39.938874 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:46:39.938860 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb\": container with ID starting with f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb not found: ID does not exist" containerID="f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb" Jul 24 13:46:39.938914 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.938882 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb"} err="failed to get container status \"f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb\": rpc error: code = NotFound desc = could not find container \"f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb\": container with ID starting with f422668c39ab20c220c888713e6d568ec731c1e1ae854abc8e9ed335d37a87eb not found: ID does not exist" Jul 24 13:46:39.943441 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:39.943421 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-50b298-openshift-default-7c54f59658-k55tg"] Jul 24 13:46:41.895721 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:41.895688 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" path="/var/lib/kubelet/pods/a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e/volumes" Jul 24 13:46:56.236008 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:56.235921 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7"] Jul 24 13:46:56.236368 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:56.236138 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" containerID="cri-o://c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c" gracePeriod=30 Jul 24 13:46:57.154812 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.154778 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg"] Jul 24 13:46:57.155150 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.155137 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" Jul 24 13:46:57.155201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.155151 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" Jul 24 13:46:57.155201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.155161 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" Jul 24 13:46:57.155201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.155167 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" Jul 24 13:46:57.155322 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.155261 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" Jul 24 13:46:57.160177 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.160158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.162662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.162646 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-502742-openshift-default-dockercfg-v4g7v\"" Jul 24 13:46:57.175428 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.175404 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg"] Jul 24 13:46:57.196473 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-socket\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.196617 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-credential-socket\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.196617 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196608 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-data\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.196767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-token\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.196767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-envoy\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.196767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e199d45-2927-4b19-a1f6-1ff59bd99085-istiod-ca-cert\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.196767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-certs\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.197044 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-podinfo\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.197044 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.196821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6mns\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-kube-api-access-g6mns\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.297675 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.297639 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-socket\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.297688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-credential-socket\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.297787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-data\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.297820 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-token\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.297837 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-envoy\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.297852 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e199d45-2927-4b19-a1f6-1ff59bd99085-istiod-ca-cert\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-certs\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298070 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-podinfo\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298097 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6mns\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-kube-api-access-g6mns\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298505 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298110 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-socket\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298505 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298130 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-credential-socket\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298505 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298193 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-data\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298505 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298419 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-certs\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.298709 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.298537 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e199d45-2927-4b19-a1f6-1ff59bd99085-istiod-ca-cert\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.300192 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.300167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-envoy\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.300469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.300443 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-podinfo\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.305457 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.305436 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-token\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.305746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.305729 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6mns\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-kube-api-access-g6mns\") pod \"e2e-api-a-502742-openshift-default-f67679cd4-kt9tg\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.470462 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.470426 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:57.605310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.605275 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg"] Jul 24 13:46:57.610788 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.610754 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:46:57.610927 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.610834 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:46:57.610927 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:57.610879 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:46:58.017371 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:58.017331 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" event={"ID":"6e199d45-2927-4b19-a1f6-1ff59bd99085","Type":"ContainerStarted","Data":"99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429"} Jul 24 13:46:58.017605 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:58.017391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" event={"ID":"6e199d45-2927-4b19-a1f6-1ff59bd99085","Type":"ContainerStarted","Data":"068fe592735cdd7fa000ccdb6ad44cd1def7f055d61eb2f10408055031cf61ba"} Jul 24 13:46:58.038106 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:58.038053 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podStartSLOduration=1.038037165 podStartE2EDuration="1.038037165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:46:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:46:58.036304335 +0000 UTC m=+2244.635549560" watchObservedRunningTime="2026-07-24 13:46:58.038037165 +0000 UTC m=+2244.637282376" Jul 24 13:46:58.471147 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:58.471116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:58.476126 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:58.476105 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:59.023620 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:59.023593 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:46:59.024634 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:46:59.024617 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:01.490636 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.490609 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:47:01.536973 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.536902 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-istio-token\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.536980 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2687f56-9671-44be-ba87-4d34139398f1-istiod-ca-cert\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537023 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-credential-socket\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537068 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvszh\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-kube-api-access-gvszh\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537120 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537101 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-data\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537130 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-envoy\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2687f56-9671-44be-ba87-4d34139398f1-istio-podinfo\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537204 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-certs\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.537347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537245 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-socket\") pod \"e2687f56-9671-44be-ba87-4d34139398f1\" (UID: \"e2687f56-9671-44be-ba87-4d34139398f1\") " Jul 24 13:47:01.538037 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537790 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:01.538037 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537726 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2687f56-9671-44be-ba87-4d34139398f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:47:01.538037 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537896 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:01.538037 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.537951 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:01.538309 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.538246 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:01.539331 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.539300 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:47:01.539685 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.539666 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2687f56-9671-44be-ba87-4d34139398f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:47:01.539906 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.539871 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-kube-api-access-gvszh" (OuterVolumeSpecName: "kube-api-access-gvszh") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "kube-api-access-gvszh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:47:01.540147 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.540129 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2687f56-9671-44be-ba87-4d34139398f1" (UID: "e2687f56-9671-44be-ba87-4d34139398f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:01.638433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638404 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638426 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638433 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638437 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638445 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2687f56-9671-44be-ba87-4d34139398f1-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638453 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638461 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvszh\" (UniqueName: \"kubernetes.io/projected/e2687f56-9671-44be-ba87-4d34139398f1-kube-api-access-gvszh\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638469 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638497 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2687f56-9671-44be-ba87-4d34139398f1-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:01.638683 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:01.638505 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2687f56-9671-44be-ba87-4d34139398f1-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:02.043462 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.043425 2562 generic.go:356] "Generic (PLEG): container finished" podID="e2687f56-9671-44be-ba87-4d34139398f1" containerID="c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c" exitCode=0 Jul 24 13:47:02.043642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.043487 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" Jul 24 13:47:02.043642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.043498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" event={"ID":"e2687f56-9671-44be-ba87-4d34139398f1","Type":"ContainerDied","Data":"c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c"} Jul 24 13:47:02.043642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.043538 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7" event={"ID":"e2687f56-9671-44be-ba87-4d34139398f1","Type":"ContainerDied","Data":"a7ef54ee2803453e7fb6eb1473d41fa516f178ee22744308f5914de159a44b75"} Jul 24 13:47:02.043642 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.043574 2562 scope.go:117] "RemoveContainer" containerID="c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c" Jul 24 13:47:02.052565 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.052536 2562 scope.go:117] "RemoveContainer" containerID="62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f" Jul 24 13:47:02.063898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.063871 2562 scope.go:117] "RemoveContainer" containerID="c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c" Jul 24 13:47:02.064229 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:47:02.064210 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c\": container with ID starting with c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c not found: ID does not exist" containerID="c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c" Jul 24 13:47:02.064323 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.064239 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c"} err="failed to get container status \"c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c\": rpc error: code = NotFound desc = could not find container \"c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c\": container with ID starting with c8ed95c08f93343ebc729ddd7cef34cd1cb80599f36a50e89708bfa4f6059e7c not found: ID does not exist" Jul 24 13:47:02.064323 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.064259 2562 scope.go:117] "RemoveContainer" containerID="62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f" Jul 24 13:47:02.064514 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:47:02.064491 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f\": container with ID starting with 62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f not found: ID does not exist" containerID="62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f" Jul 24 13:47:02.064609 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.064520 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f"} err="failed to get container status \"62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f\": rpc error: code = NotFound desc = could not find container \"62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f\": container with ID starting with 62f8a578c37289cd10abf0380bc56a4ff1b6046465b52c62cd304da8aa9c345f not found: ID does not exist" Jul 24 13:47:02.064689 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.064666 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7"] Jul 24 13:47:02.067970 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:02.067950 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7918ee-openshift-default-5d9d95df8b-dztx7"] Jul 24 13:47:03.896210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:03.896181 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2687f56-9671-44be-ba87-4d34139398f1" path="/var/lib/kubelet/pods/e2687f56-9671-44be-ba87-4d34139398f1/volumes" Jul 24 13:47:05.373627 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.373600 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6"] Jul 24 13:47:05.374122 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.374103 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" Jul 24 13:47:05.374203 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.374124 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" Jul 24 13:47:05.374203 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.374137 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" Jul 24 13:47:05.374203 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.374145 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" Jul 24 13:47:05.374350 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.374245 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" Jul 24 13:47:05.374350 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.374259 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a81dfb27-ada9-4b00-b2c2-fa4656d2ab7e" containerName="istio-proxy" Jul 24 13:47:05.378680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.378661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.381040 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.381023 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-799cec-openshift-default-dockercfg-q5t25\"" Jul 24 13:47:05.390478 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.390454 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6"] Jul 24 13:47:05.473376 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.473340 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-certs\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.473690 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.473668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-socket\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.473918 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.473883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ef35ae-57cf-487b-9c64-22756376033e-istiod-ca-cert\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.474034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.473935 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-istio-token\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.474034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.473967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ef35ae-57cf-487b-9c64-22756376033e-istio-podinfo\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.474034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.474013 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-envoy\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.474188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.474044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdb8z\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-kube-api-access-bdb8z\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.474188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.474089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-credential-socket\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.474188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.474182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-data\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575330 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575297 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-certs\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575484 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-socket\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575484 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ef35ae-57cf-487b-9c64-22756376033e-istiod-ca-cert\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575484 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-istio-token\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575693 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ef35ae-57cf-487b-9c64-22756376033e-istio-podinfo\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575693 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-envoy\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575796 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdb8z\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-kube-api-access-bdb8z\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575796 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575745 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-socket\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575796 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-credential-socket\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575846 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-data\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.575956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.575872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-certs\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.576153 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.576131 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ef35ae-57cf-487b-9c64-22756376033e-istiod-ca-cert\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.576153 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.576145 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-credential-socket\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.576262 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.576177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-data\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.577880 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.577853 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ef35ae-57cf-487b-9c64-22756376033e-istio-podinfo\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.578043 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.578025 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-envoy\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.584585 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.584535 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdb8z\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-kube-api-access-bdb8z\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.584901 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.584882 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-istio-token\") pod \"e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.688776 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.688750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:05.829959 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.829935 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6"] Jul 24 13:47:05.831725 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:47:05.831699 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ef35ae_57cf_487b_9c64_22756376033e.slice/crio-f0e91bd2272f09d4ead0b894093f13d87b4caed2b343ea1986615f8806bd0b10 WatchSource:0}: Error finding container f0e91bd2272f09d4ead0b894093f13d87b4caed2b343ea1986615f8806bd0b10: Status 404 returned error can't find the container with id f0e91bd2272f09d4ead0b894093f13d87b4caed2b343ea1986615f8806bd0b10 Jul 24 13:47:05.833876 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.833845 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:05.833959 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.833908 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:05.833959 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:05.833938 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:06.068123 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:06.068037 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" event={"ID":"77ef35ae-57cf-487b-9c64-22756376033e","Type":"ContainerStarted","Data":"42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0"} Jul 24 13:47:06.068123 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:06.068100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" event={"ID":"77ef35ae-57cf-487b-9c64-22756376033e","Type":"ContainerStarted","Data":"f0e91bd2272f09d4ead0b894093f13d87b4caed2b343ea1986615f8806bd0b10"} Jul 24 13:47:06.092408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:06.092358 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podStartSLOduration=1.092344337 podStartE2EDuration="1.092344337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:47:06.090905408 +0000 UTC m=+2252.690150661" watchObservedRunningTime="2026-07-24 13:47:06.092344337 +0000 UTC m=+2252.691589548" Jul 24 13:47:06.689450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:06.689416 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:06.694381 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:06.694356 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:07.073873 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:07.073793 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:07.074833 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:07.074815 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:14.811099 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:14.810110 2562 patch_prober.go:28] interesting pod/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:36798->10.133.0.53:15021: read: connection reset by peer" start-of-body= Jul 24 13:47:14.811099 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:14.810181 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:36798->10.133.0.53:15021: read: connection reset by peer" Jul 24 13:47:15.130178 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:15.130084 2562 generic.go:356] "Generic (PLEG): container finished" podID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerID="99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429" exitCode=0 Jul 24 13:47:15.130178 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:15.130138 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" event={"ID":"6e199d45-2927-4b19-a1f6-1ff59bd99085","Type":"ContainerDied","Data":"99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429"} Jul 24 13:47:15.130584 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:15.130569 2562 scope.go:117] "RemoveContainer" containerID="99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429" Jul 24 13:47:15.131634 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:15.131605 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:15.131702 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:15.131683 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:15.131742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:15.131719 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:16.137111 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:16.137077 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" event={"ID":"6e199d45-2927-4b19-a1f6-1ff59bd99085","Type":"ContainerStarted","Data":"aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163"} Jul 24 13:47:16.470961 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:16.470918 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:16.471212 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:16.471182 2562 patch_prober.go:28] interesting pod/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 13:47:16.471335 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:16.471242 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 13:47:17.471782 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:17.471741 2562 patch_prober.go:28] interesting pod/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 13:47:17.472236 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:17.471816 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 13:47:18.470967 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:18.470931 2562 patch_prober.go:28] interesting pod/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 13:47:18.471145 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:18.470985 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 13:47:19.471298 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:19.471257 2562 patch_prober.go:28] interesting pod/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 24 13:47:19.471719 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:19.471317 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 24 13:47:20.474703 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:20.474669 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:20.475177 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:20.475035 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:20.475775 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:20.475756 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:23.115711 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:23.115671 2562 patch_prober.go:28] interesting pod/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 13:47:23.116095 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:23.115741 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 13:47:25.183791 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:25.183760 2562 generic.go:356] "Generic (PLEG): container finished" podID="77ef35ae-57cf-487b-9c64-22756376033e" containerID="42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0" exitCode=0 Jul 24 13:47:25.184310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:25.183817 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" event={"ID":"77ef35ae-57cf-487b-9c64-22756376033e","Type":"ContainerDied","Data":"42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0"} Jul 24 13:47:25.184310 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:25.184295 2562 scope.go:117] "RemoveContainer" containerID="42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0" Jul 24 13:47:25.185352 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:25.185322 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:25.185448 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:25.185402 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:25.185511 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:25.185446 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:47:26.194646 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:26.194605 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" event={"ID":"77ef35ae-57cf-487b-9c64-22756376033e","Type":"ContainerStarted","Data":"dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06"} Jul 24 13:47:26.688881 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:26.688844 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:26.689287 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:26.689255 2562 patch_prober.go:28] interesting pod/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 13:47:26.689402 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:26.689311 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 13:47:27.689992 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:27.689953 2562 patch_prober.go:28] interesting pod/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 13:47:27.690519 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:27.690015 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 13:47:28.689159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:28.689114 2562 patch_prober.go:28] interesting pod/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 13:47:28.689373 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:28.689179 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 13:47:29.689414 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:29.689375 2562 patch_prober.go:28] interesting pod/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 13:47:29.690034 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:29.689999 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 13:47:30.693437 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:30.693403 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:30.693922 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:30.693902 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:30.694622 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:30.694602 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:47:47.424292 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:47.424261 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg"] Jul 24 13:47:47.424736 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:47.424477 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" containerID="cri-o://aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163" gracePeriod=30 Jul 24 13:47:50.477529 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:50.477496 2562 patch_prober.go:28] interesting pod/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 13:47:50.477921 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:50.477584 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:47:52.678943 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.678919 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:52.715783 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715714 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e199d45-2927-4b19-a1f6-1ff59bd99085-istiod-ca-cert\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715783 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715748 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-token\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715987 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715789 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6mns\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-kube-api-access-g6mns\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715987 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715820 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-data\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715987 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715842 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-certs\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715987 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715871 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-credential-socket\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715987 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715917 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-podinfo\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.715987 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.715951 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-envoy\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.716319 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716005 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-socket\") pod \"6e199d45-2927-4b19-a1f6-1ff59bd99085\" (UID: \"6e199d45-2927-4b19-a1f6-1ff59bd99085\") " Jul 24 13:47:52.716319 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716204 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e199d45-2927-4b19-a1f6-1ff59bd99085-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:47:52.716420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716393 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e199d45-2927-4b19-a1f6-1ff59bd99085-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.716420 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716401 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:52.716591 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716562 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:52.716662 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716629 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:52.716847 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.716823 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:52.718135 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.718103 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:47:52.718465 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.718440 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-kube-api-access-g6mns" (OuterVolumeSpecName: "kube-api-access-g6mns") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "kube-api-access-g6mns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:47:52.718565 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.718484 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:47:52.718669 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.718652 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e199d45-2927-4b19-a1f6-1ff59bd99085" (UID: "6e199d45-2927-4b19-a1f6-1ff59bd99085"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:47:52.817002 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.816975 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817002 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.816996 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817002 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.817005 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.817014 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.817023 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.817034 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e199d45-2927-4b19-a1f6-1ff59bd99085-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.817042 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:52.817210 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:52.817050 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6mns\" (UniqueName: \"kubernetes.io/projected/6e199d45-2927-4b19-a1f6-1ff59bd99085-kube-api-access-g6mns\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:47:53.338693 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.338650 2562 generic.go:356] "Generic (PLEG): container finished" podID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerID="aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163" exitCode=0 Jul 24 13:47:53.338868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.338725 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" Jul 24 13:47:53.338868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.338735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" event={"ID":"6e199d45-2927-4b19-a1f6-1ff59bd99085","Type":"ContainerDied","Data":"aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163"} Jul 24 13:47:53.338868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.338806 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg" event={"ID":"6e199d45-2927-4b19-a1f6-1ff59bd99085","Type":"ContainerDied","Data":"068fe592735cdd7fa000ccdb6ad44cd1def7f055d61eb2f10408055031cf61ba"} Jul 24 13:47:53.338868 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.338839 2562 scope.go:117] "RemoveContainer" containerID="aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163" Jul 24 13:47:53.348350 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.348331 2562 scope.go:117] "RemoveContainer" containerID="99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429" Jul 24 13:47:53.356419 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.356400 2562 scope.go:117] "RemoveContainer" containerID="aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163" Jul 24 13:47:53.356768 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:47:53.356750 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163\": container with ID starting with aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163 not found: ID does not exist" containerID="aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163" Jul 24 13:47:53.356832 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.356775 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163"} err="failed to get container status \"aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163\": rpc error: code = NotFound desc = could not find container \"aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163\": container with ID starting with aeae1412da65eb50198a27ddc5f93edbcb8de33d147a9432630a6db4b2552163 not found: ID does not exist" Jul 24 13:47:53.356832 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.356793 2562 scope.go:117] "RemoveContainer" containerID="99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429" Jul 24 13:47:53.357018 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:47:53.357004 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429\": container with ID starting with 99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429 not found: ID does not exist" containerID="99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429" Jul 24 13:47:53.357063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.357021 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429"} err="failed to get container status \"99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429\": rpc error: code = NotFound desc = could not find container \"99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429\": container with ID starting with 99bd600e6181dc24bf72baa1a770a8e353dee74bf000af552e31a71573f41429 not found: ID does not exist" Jul 24 13:47:53.360661 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.360641 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg"] Jul 24 13:47:53.367245 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.367226 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-502742-openshift-default-f67679cd4-kt9tg"] Jul 24 13:47:53.899531 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:47:53.899504 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" path="/var/lib/kubelet/pods/6e199d45-2927-4b19-a1f6-1ff59bd99085/volumes" Jul 24 13:48:14.378664 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:14.378631 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6"] Jul 24 13:48:14.379278 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:14.378829 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" containerID="cri-o://dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06" gracePeriod=30 Jul 24 13:48:15.329837 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.329791 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d"] Jul 24 13:48:15.330231 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.330214 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" Jul 24 13:48:15.330319 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.330233 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" Jul 24 13:48:15.330319 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.330262 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" Jul 24 13:48:15.330319 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.330271 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" Jul 24 13:48:15.330475 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.330370 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2687f56-9671-44be-ba87-4d34139398f1" containerName="istio-proxy" Jul 24 13:48:15.330475 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.330382 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" Jul 24 13:48:15.333407 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.333385 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.335650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.335630 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-934481-openshift-default-dockercfg-79wnw\"" Jul 24 13:48:15.344661 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.344640 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d"] Jul 24 13:48:15.420896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.420852 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-certs\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.420896 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.420896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-socket\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421024 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-credential-socket\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqjcm\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-kube-api-access-nqjcm\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istiod-ca-cert\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421144 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-token\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421247 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-podinfo\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421272 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-envoy\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.421429 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.421289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-data\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522219 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522186 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-credential-socket\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqjcm\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-kube-api-access-nqjcm\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istiod-ca-cert\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522270 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-token\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-podinfo\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522333 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-envoy\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-data\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522390 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-certs\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522417 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-socket\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522670 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-credential-socket\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-data\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.522877 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-socket\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.523071 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.522914 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-certs\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.523175 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.523158 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istiod-ca-cert\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.524706 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.524683 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-envoy\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.524921 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.524899 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-podinfo\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.536991 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.536970 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqjcm\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-kube-api-access-nqjcm\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.537159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.537140 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-token\") pod \"e2e-shared-a-934481-openshift-default-7df6768488-vz89d\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.644252 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.644158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:15.697003 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.696910 2562 patch_prober.go:28] interesting pod/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 13:48:15.697173 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.696979 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:48:15.776475 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.776451 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d"] Jul 24 13:48:15.777888 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:48:15.777861 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a148cc6_d3d7_45b6_91ee_1d2eee9b4974.slice/crio-92276421d55b4028fc991316a90e56e6dfc12c1f63577e5fe81d73ecd0255f50 WatchSource:0}: Error finding container 92276421d55b4028fc991316a90e56e6dfc12c1f63577e5fe81d73ecd0255f50: Status 404 returned error can't find the container with id 92276421d55b4028fc991316a90e56e6dfc12c1f63577e5fe81d73ecd0255f50 Jul 24 13:48:15.780024 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.779998 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:15.780091 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.780055 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:15.780091 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:15.780082 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:16.464768 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:16.464723 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" event={"ID":"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974","Type":"ContainerStarted","Data":"5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d"} Jul 24 13:48:16.465148 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:16.464824 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" event={"ID":"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974","Type":"ContainerStarted","Data":"92276421d55b4028fc991316a90e56e6dfc12c1f63577e5fe81d73ecd0255f50"} Jul 24 13:48:16.486257 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:16.486201 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podStartSLOduration=1.486186992 podStartE2EDuration="1.486186992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:16.484335932 +0000 UTC m=+2323.083581142" watchObservedRunningTime="2026-07-24 13:48:16.486186992 +0000 UTC m=+2323.085432203" Jul 24 13:48:16.645126 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:16.645097 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:16.650016 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:16.649991 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:17.469823 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:17.469791 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:17.470767 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:17.470746 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:19.636312 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.636287 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:48:19.762185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762153 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-socket\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762185 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762187 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-istio-token\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762430 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762205 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-data\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762430 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762242 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ef35ae-57cf-487b-9c64-22756376033e-istiod-ca-cert\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762430 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762325 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-certs\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762430 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762357 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-credential-socket\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762430 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762385 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-envoy\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762431 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdb8z\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-kube-api-access-bdb8z\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762442 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:48:19.762732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762502 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ef35ae-57cf-487b-9c64-22756376033e-istio-podinfo\") pod \"77ef35ae-57cf-487b-9c64-22756376033e\" (UID: \"77ef35ae-57cf-487b-9c64-22756376033e\") " Jul 24 13:48:19.762732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762690 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:48:19.762732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762711 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:48:19.762732 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762722 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77ef35ae-57cf-487b-9c64-22756376033e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:48:19.762969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762886 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.762969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762906 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.762969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762921 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.762969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.762936 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ef35ae-57cf-487b-9c64-22756376033e-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.763201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.763177 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-data" (OuterVolumeSpecName: "istio-data") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:48:19.764469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.764440 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-istio-token" (OuterVolumeSpecName: "istio-token") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:48:19.764630 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.764497 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77ef35ae-57cf-487b-9c64-22756376033e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:48:19.764630 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.764513 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:48:19.764731 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.764713 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-kube-api-access-bdb8z" (OuterVolumeSpecName: "kube-api-access-bdb8z") pod "77ef35ae-57cf-487b-9c64-22756376033e" (UID: "77ef35ae-57cf-487b-9c64-22756376033e"). InnerVolumeSpecName "kube-api-access-bdb8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:48:19.864389 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.864357 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ef35ae-57cf-487b-9c64-22756376033e-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.864389 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.864385 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.864389 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.864394 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.864633 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.864403 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ef35ae-57cf-487b-9c64-22756376033e-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:19.864633 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:19.864411 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdb8z\" (UniqueName: \"kubernetes.io/projected/77ef35ae-57cf-487b-9c64-22756376033e-kube-api-access-bdb8z\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:48:20.492459 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.492421 2562 generic.go:356] "Generic (PLEG): container finished" podID="77ef35ae-57cf-487b-9c64-22756376033e" containerID="dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06" exitCode=0 Jul 24 13:48:20.492636 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.492479 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" Jul 24 13:48:20.492636 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.492478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" event={"ID":"77ef35ae-57cf-487b-9c64-22756376033e","Type":"ContainerDied","Data":"dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06"} Jul 24 13:48:20.492636 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.492626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6" event={"ID":"77ef35ae-57cf-487b-9c64-22756376033e","Type":"ContainerDied","Data":"f0e91bd2272f09d4ead0b894093f13d87b4caed2b343ea1986615f8806bd0b10"} Jul 24 13:48:20.492780 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.492649 2562 scope.go:117] "RemoveContainer" containerID="dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06" Jul 24 13:48:20.501740 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.501487 2562 scope.go:117] "RemoveContainer" containerID="42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0" Jul 24 13:48:20.510076 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.510059 2562 scope.go:117] "RemoveContainer" containerID="dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06" Jul 24 13:48:20.510318 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:48:20.510300 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06\": container with ID starting with dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06 not found: ID does not exist" containerID="dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06" Jul 24 13:48:20.510368 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.510326 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06"} err="failed to get container status \"dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06\": rpc error: code = NotFound desc = could not find container \"dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06\": container with ID starting with dd92e5974c27535d9a9efee8922cbba39fa8e3f5c081631c6c003c648ca99d06 not found: ID does not exist" Jul 24 13:48:20.510368 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.510347 2562 scope.go:117] "RemoveContainer" containerID="42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0" Jul 24 13:48:20.510610 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:48:20.510582 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0\": container with ID starting with 42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0 not found: ID does not exist" containerID="42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0" Jul 24 13:48:20.510707 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.510616 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0"} err="failed to get container status \"42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0\": rpc error: code = NotFound desc = could not find container \"42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0\": container with ID starting with 42d6e94d963958abab4c6c178984593ccfbe42972c15d66d88621e554156d7f0 not found: ID does not exist" Jul 24 13:48:20.510962 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.510943 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6"] Jul 24 13:48:20.516139 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:20.516120 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-799cec-openshift-default-5bb94c8d8b-vbgp6"] Jul 24 13:48:21.896284 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:21.896251 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77ef35ae-57cf-487b-9c64-22756376033e" path="/var/lib/kubelet/pods/77ef35ae-57cf-487b-9c64-22756376033e/volumes" Jul 24 13:48:23.658303 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.658265 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj"] Jul 24 13:48:23.658685 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.658645 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" Jul 24 13:48:23.658685 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.658656 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" Jul 24 13:48:23.658827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.658721 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e199d45-2927-4b19-a1f6-1ff59bd99085" containerName="istio-proxy" Jul 24 13:48:23.658827 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.658730 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" Jul 24 13:48:23.663006 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.662986 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.665844 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.665825 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0a0df7-openshift-default-dockercfg-nt2fq\"" Jul 24 13:48:23.683246 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.683205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj"] Jul 24 13:48:23.801958 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.801923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-podinfo\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802151 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.801970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-token\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802151 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.801991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-data\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802151 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.802075 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-certs\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802151 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.802127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-socket\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802376 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.802169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-envoy\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802376 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.802232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-228bt\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-kube-api-access-228bt\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802376 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.802262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-credential-socket\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.802376 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.802293 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91086b03-c87c-44b2-b8bd-e2bac1294164-istiod-ca-cert\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.903663 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-certs\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.903808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-socket\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.903808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903694 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-envoy\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.903808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903724 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-228bt\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-kube-api-access-228bt\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.903808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-credential-socket\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.903808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91086b03-c87c-44b2-b8bd-e2bac1294164-istiod-ca-cert\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904083 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903833 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-podinfo\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904083 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-token\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904083 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.903905 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-data\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904083 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.904054 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-certs\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904083 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.904065 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-socket\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904388 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.904347 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-credential-socket\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.904439 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-data\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.904559 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.904526 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91086b03-c87c-44b2-b8bd-e2bac1294164-istiod-ca-cert\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.906146 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.906125 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-envoy\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.906425 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.906403 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-podinfo\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.911346 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.911295 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-token\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.911431 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.911390 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-228bt\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-kube-api-access-228bt\") pod \"e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:23.973442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:23.973414 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:24.122687 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.122661 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj"] Jul 24 13:48:24.123882 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:48:24.123851 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91086b03_c87c_44b2_b8bd_e2bac1294164.slice/crio-15aff688b07e4d02f99166ebe5ae9ff5745e4a58234222ef8bdb272712eb2d3a WatchSource:0}: Error finding container 15aff688b07e4d02f99166ebe5ae9ff5745e4a58234222ef8bdb272712eb2d3a: Status 404 returned error can't find the container with id 15aff688b07e4d02f99166ebe5ae9ff5745e4a58234222ef8bdb272712eb2d3a Jul 24 13:48:24.126067 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.126036 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:24.126158 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.126103 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:24.126158 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.126129 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:24.515470 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.515422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" event={"ID":"91086b03-c87c-44b2-b8bd-e2bac1294164","Type":"ContainerStarted","Data":"587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200"} Jul 24 13:48:24.515669 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.515495 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" event={"ID":"91086b03-c87c-44b2-b8bd-e2bac1294164","Type":"ContainerStarted","Data":"15aff688b07e4d02f99166ebe5ae9ff5745e4a58234222ef8bdb272712eb2d3a"} Jul 24 13:48:24.538480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.538426 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" podStartSLOduration=1.5384078479999999 podStartE2EDuration="1.538407848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:48:24.534337331 +0000 UTC m=+2331.133582555" watchObservedRunningTime="2026-07-24 13:48:24.538407848 +0000 UTC m=+2331.137653061" Jul 24 13:48:24.973912 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:24.973887 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:25.978599 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:25.978564 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:26.527019 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:26.526986 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:26.528290 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:26.528259 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:32.472061 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.472023 2562 patch_prober.go:28] interesting pod/e2e-shared-a-934481-openshift-default-7df6768488-vz89d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 13:48:32.472458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.472096 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 13:48:32.559569 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.559515 2562 generic.go:356] "Generic (PLEG): container finished" podID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerID="5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d" exitCode=0 Jul 24 13:48:32.559728 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.559581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" event={"ID":"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974","Type":"ContainerDied","Data":"5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d"} Jul 24 13:48:32.559988 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.559974 2562 scope.go:117] "RemoveContainer" containerID="5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d" Jul 24 13:48:32.561029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.561004 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:32.561101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.561069 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:32.561149 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:32.561100 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:33.567747 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:33.567709 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" event={"ID":"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974","Type":"ContainerStarted","Data":"2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6"} Jul 24 13:48:33.645180 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:33.645145 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:33.645349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:33.645326 2562 patch_prober.go:28] interesting pod/e2e-shared-a-934481-openshift-default-7df6768488-vz89d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 13:48:33.645395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:33.645375 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 13:48:34.645450 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:34.645410 2562 patch_prober.go:28] interesting pod/e2e-shared-a-934481-openshift-default-7df6768488-vz89d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 13:48:34.645898 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:34.645464 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 13:48:35.644973 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:35.644886 2562 patch_prober.go:28] interesting pod/e2e-shared-a-934481-openshift-default-7df6768488-vz89d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 13:48:35.644973 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:35.644956 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 13:48:36.645279 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:36.645241 2562 patch_prober.go:28] interesting pod/e2e-shared-a-934481-openshift-default-7df6768488-vz89d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 13:48:36.645749 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:36.645298 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 13:48:37.644795 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:37.644764 2562 patch_prober.go:28] interesting pod/e2e-shared-a-934481-openshift-default-7df6768488-vz89d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 13:48:37.644961 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:37.644823 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 13:48:38.599590 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.599473 2562 generic.go:356] "Generic (PLEG): container finished" podID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerID="587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200" exitCode=0 Jul 24 13:48:38.599590 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.599566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" event={"ID":"91086b03-c87c-44b2-b8bd-e2bac1294164","Type":"ContainerDied","Data":"587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200"} Jul 24 13:48:38.600032 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.599955 2562 scope.go:117] "RemoveContainer" containerID="587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200" Jul 24 13:48:38.600937 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.600909 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:38.601038 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.600991 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:38.601101 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.601039 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:48:38.649968 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.649940 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:38.650480 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.650457 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:38.651271 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:38.651254 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:48:39.611295 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:39.611258 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" event={"ID":"91086b03-c87c-44b2-b8bd-e2bac1294164","Type":"ContainerStarted","Data":"77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98"} Jul 24 13:48:39.973727 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:39.973691 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:39.974693 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:39.974668 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 13:48:39.974829 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:39.974724 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 13:48:40.974004 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:40.973973 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 13:48:40.974366 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:40.974025 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 13:48:41.532991 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:41.532950 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:41.974191 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:41.974156 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 13:48:41.974580 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:41.974214 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 13:48:42.974145 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:42.974106 2562 patch_prober.go:28] interesting pod/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 13:48:42.974349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:42.974167 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 13:48:43.977874 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:43.977843 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:48:43.978837 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:48:43.978819 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 13:49:19.123833 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.123791 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s"] Jul 24 13:49:19.124518 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.124252 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" Jul 24 13:49:19.124518 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.124266 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" Jul 24 13:49:19.124518 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.124327 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ef35ae-57cf-487b-9c64-22756376033e" containerName="istio-proxy" Jul 24 13:49:19.127490 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.127476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.130005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.129984 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a0df7\"/\"openshift-service-ca.crt\"" Jul 24 13:49:19.130782 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.130764 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a0df7\"/\"default-dockercfg-md425\"" Jul 24 13:49:19.130893 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.130875 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0a0df7\"/\"kube-root-ca.crt\"" Jul 24 13:49:19.130975 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.130875 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0a0df7\"/\"auth-test-model-0a0df7-kserve-self-signed-certs\"" Jul 24 13:49:19.135815 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.135791 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s"] Jul 24 13:49:19.200619 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.200595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-model-cache\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.200773 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.200627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-home\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.200773 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.200749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8jvf\" (UniqueName: \"kubernetes.io/projected/f65f3409-8ef5-46ce-9189-ba34d885eb0c-kube-api-access-w8jvf\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.200897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.200794 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tls-certs\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.200897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.200835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-dshm\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.200897 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.200878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tmp-dir\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301332 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301307 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8jvf\" (UniqueName: \"kubernetes.io/projected/f65f3409-8ef5-46ce-9189-ba34d885eb0c-kube-api-access-w8jvf\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301341 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tls-certs\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-dshm\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tmp-dir\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-model-cache\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301469 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301468 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-home\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.301934 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301909 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-home\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.302005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301970 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tmp-dir\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.302005 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.301994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-model-cache\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.303680 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.303662 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-dshm\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.303923 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.303902 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tls-certs\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.310138 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.310117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8jvf\" (UniqueName: \"kubernetes.io/projected/f65f3409-8ef5-46ce-9189-ba34d885eb0c-kube-api-access-w8jvf\") pod \"auth-test-model-0a0df7-kserve-6c75695dcf-f652s\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.437192 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.437161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:19.566842 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.566812 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s"] Jul 24 13:49:19.568288 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:49:19.568266 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf65f3409_8ef5_46ce_9189_ba34d885eb0c.slice/crio-695a80aa42bb687e111ef688f87211700071f612f84052fd81efb4200fa65594 WatchSource:0}: Error finding container 695a80aa42bb687e111ef688f87211700071f612f84052fd81efb4200fa65594: Status 404 returned error can't find the container with id 695a80aa42bb687e111ef688f87211700071f612f84052fd81efb4200fa65594 Jul 24 13:49:19.814920 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.814839 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" event={"ID":"f65f3409-8ef5-46ce-9189-ba34d885eb0c","Type":"ContainerStarted","Data":"4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a"} Jul 24 13:49:19.814920 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.814878 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" event={"ID":"f65f3409-8ef5-46ce-9189-ba34d885eb0c","Type":"ContainerStarted","Data":"695a80aa42bb687e111ef688f87211700071f612f84052fd81efb4200fa65594"} Jul 24 13:49:19.832492 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:19.832442 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" podStartSLOduration=0.83242752 podStartE2EDuration="832.42752ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:49:19.830948175 +0000 UTC m=+2386.430193387" watchObservedRunningTime="2026-07-24 13:49:19.83242752 +0000 UTC m=+2386.431672730" Jul 24 13:49:29.437604 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:29.437541 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:29.438062 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:29.437620 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:29.450048 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:29.450022 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:49:29.877183 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:49:29.877152 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:51:18.205025 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:18.204990 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s"] Jul 24 13:51:33.223596 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.223538 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" podUID="f65f3409-8ef5-46ce-9189-ba34d885eb0c" containerName="main" containerID="cri-o://4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a" gracePeriod=45 Jul 24 13:51:33.465347 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.465322 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:51:33.495102 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495045 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-home\") pod \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " Jul 24 13:51:33.495102 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495080 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tls-certs\") pod \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " Jul 24 13:51:33.495277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495131 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tmp-dir\") pod \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " Jul 24 13:51:33.495277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495150 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-model-cache\") pod \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " Jul 24 13:51:33.495277 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495169 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-dshm\") pod \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " Jul 24 13:51:33.495440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495272 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8jvf\" (UniqueName: \"kubernetes.io/projected/f65f3409-8ef5-46ce-9189-ba34d885eb0c-kube-api-access-w8jvf\") pod \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\" (UID: \"f65f3409-8ef5-46ce-9189-ba34d885eb0c\") " Jul 24 13:51:33.495440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495323 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-home" (OuterVolumeSpecName: "home") pod "f65f3409-8ef5-46ce-9189-ba34d885eb0c" (UID: "f65f3409-8ef5-46ce-9189-ba34d885eb0c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:51:33.495440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495378 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f65f3409-8ef5-46ce-9189-ba34d885eb0c" (UID: "f65f3409-8ef5-46ce-9189-ba34d885eb0c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:51:33.495440 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495412 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f65f3409-8ef5-46ce-9189-ba34d885eb0c" (UID: "f65f3409-8ef5-46ce-9189-ba34d885eb0c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:51:33.495674 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495610 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tmp-dir\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:51:33.495674 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495629 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-model-cache\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:51:33.495674 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.495644 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-home\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:51:33.498132 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.498101 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f65f3409-8ef5-46ce-9189-ba34d885eb0c" (UID: "f65f3409-8ef5-46ce-9189-ba34d885eb0c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:51:33.498273 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.498195 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-dshm" (OuterVolumeSpecName: "dshm") pod "f65f3409-8ef5-46ce-9189-ba34d885eb0c" (UID: "f65f3409-8ef5-46ce-9189-ba34d885eb0c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:51:33.499486 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.499453 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f65f3409-8ef5-46ce-9189-ba34d885eb0c-kube-api-access-w8jvf" (OuterVolumeSpecName: "kube-api-access-w8jvf") pod "f65f3409-8ef5-46ce-9189-ba34d885eb0c" (UID: "f65f3409-8ef5-46ce-9189-ba34d885eb0c"). InnerVolumeSpecName "kube-api-access-w8jvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:51:33.502334 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.502309 2562 generic.go:356] "Generic (PLEG): container finished" podID="f65f3409-8ef5-46ce-9189-ba34d885eb0c" containerID="4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a" exitCode=0 Jul 24 13:51:33.502461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.502355 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" Jul 24 13:51:33.502461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.502381 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" event={"ID":"f65f3409-8ef5-46ce-9189-ba34d885eb0c","Type":"ContainerDied","Data":"4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a"} Jul 24 13:51:33.502461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.502415 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s" event={"ID":"f65f3409-8ef5-46ce-9189-ba34d885eb0c","Type":"ContainerDied","Data":"695a80aa42bb687e111ef688f87211700071f612f84052fd81efb4200fa65594"} Jul 24 13:51:33.502461 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.502434 2562 scope.go:117] "RemoveContainer" containerID="4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a" Jul 24 13:51:33.516778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.516762 2562 scope.go:117] "RemoveContainer" containerID="4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a" Jul 24 13:51:33.517042 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:51:33.517024 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a\": container with ID starting with 4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a not found: ID does not exist" containerID="4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a" Jul 24 13:51:33.517103 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.517055 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a"} err="failed to get container status \"4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a\": rpc error: code = NotFound desc = could not find container \"4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a\": container with ID starting with 4e7ea8632bad66d65fc684ce5798c31dcac6160be067e95ee7e3ca298cfb433a not found: ID does not exist" Jul 24 13:51:33.528419 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.528397 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s"] Jul 24 13:51:33.532778 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.532759 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0a0df7/auth-test-model-0a0df7-kserve-6c75695dcf-f652s"] Jul 24 13:51:33.595994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.595972 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f65f3409-8ef5-46ce-9189-ba34d885eb0c-dshm\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:51:33.595994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.595993 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8jvf\" (UniqueName: \"kubernetes.io/projected/f65f3409-8ef5-46ce-9189-ba34d885eb0c-kube-api-access-w8jvf\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:51:33.596121 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.596003 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65f3409-8ef5-46ce-9189-ba34d885eb0c-tls-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:51:33.896603 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:51:33.896518 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f65f3409-8ef5-46ce-9189-ba34d885eb0c" path="/var/lib/kubelet/pods/f65f3409-8ef5-46ce-9189-ba34d885eb0c/volumes" Jul 24 13:54:55.551811 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.551784 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9"] Jul 24 13:54:55.552327 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.552210 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f65f3409-8ef5-46ce-9189-ba34d885eb0c" containerName="main" Jul 24 13:54:55.552327 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.552228 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f65f3409-8ef5-46ce-9189-ba34d885eb0c" containerName="main" Jul 24 13:54:55.552455 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.552359 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f65f3409-8ef5-46ce-9189-ba34d885eb0c" containerName="main" Jul 24 13:54:55.555554 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.555529 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.557738 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.557719 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-e7750f-openshift-default-dockercfg-dtfm6\"" Jul 24 13:54:55.566408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.566386 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9"] Jul 24 13:54:55.595564 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-socket\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595675 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-credential-socket\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595675 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595619 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-certs\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595675 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595641 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-token\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-envoy\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595731 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7a70e-e388-48be-86f6-c67eae3e7019-istiod-ca-cert\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595807 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5r64\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-kube-api-access-q5r64\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595826 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-podinfo\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.595956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.595881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-data\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.696808 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.696767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5r64\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-kube-api-access-q5r64\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.696840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-podinfo\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.696873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-data\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.696938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-socket\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.696968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-credential-socket\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697012 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.696997 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-certs\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697283 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-token\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697342 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697310 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-envoy\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7a70e-e388-48be-86f6-c67eae3e7019-istiod-ca-cert\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697395 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697359 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-socket\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697531 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-certs\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697531 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697440 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-credential-socket\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697531 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697471 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-data\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.697956 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.697934 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7a70e-e388-48be-86f6-c67eae3e7019-istiod-ca-cert\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.699368 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.699346 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-envoy\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.699626 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.699609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-podinfo\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.704071 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.704049 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-token\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.704244 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.704228 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5r64\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-kube-api-access-q5r64\") pod \"e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.866067 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.865980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:55.999914 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:55.999871 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9"] Jul 24 13:54:56.002884 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:54:56.002855 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc7a70e_e388_48be_86f6_c67eae3e7019.slice/crio-f4b354b73a6ff90e9b5caf9f952019a762d0fedf590cfb4f41f03af5b385563e WatchSource:0}: Error finding container f4b354b73a6ff90e9b5caf9f952019a762d0fedf590cfb4f41f03af5b385563e: Status 404 returned error can't find the container with id f4b354b73a6ff90e9b5caf9f952019a762d0fedf590cfb4f41f03af5b385563e Jul 24 13:54:56.005086 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.005057 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:54:56.005193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.005135 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:54:56.005193 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.005161 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:54:56.543153 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.543120 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" event={"ID":"6bc7a70e-e388-48be-86f6-c67eae3e7019","Type":"ContainerStarted","Data":"be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b"} Jul 24 13:54:56.543331 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.543171 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" event={"ID":"6bc7a70e-e388-48be-86f6-c67eae3e7019","Type":"ContainerStarted","Data":"f4b354b73a6ff90e9b5caf9f952019a762d0fedf590cfb4f41f03af5b385563e"} Jul 24 13:54:56.563127 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.563081 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podStartSLOduration=1.563066911 podStartE2EDuration="1.563066911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:54:56.56118855 +0000 UTC m=+2723.160433761" watchObservedRunningTime="2026-07-24 13:54:56.563066911 +0000 UTC m=+2723.162312122" Jul 24 13:54:56.867088 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:56.867010 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:57.871964 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:57.871934 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:58.552736 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:58.552699 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:54:58.553609 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:54:58.553588 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:55:14.593719 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:14.593677 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 13:55:14.594156 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:14.593751 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 13:55:15.646742 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:15.646712 2562 generic.go:356] "Generic (PLEG): container finished" podID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerID="be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b" exitCode=0 Jul 24 13:55:15.647270 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:15.646790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" event={"ID":"6bc7a70e-e388-48be-86f6-c67eae3e7019","Type":"ContainerDied","Data":"be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b"} Jul 24 13:55:15.647335 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:15.647300 2562 scope.go:117] "RemoveContainer" containerID="be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b" Jul 24 13:55:15.648444 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:15.648420 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:15.648510 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:15.648485 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:15.648571 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:15.648516 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 24 13:55:16.653239 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:16.653204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" event={"ID":"6bc7a70e-e388-48be-86f6-c67eae3e7019","Type":"ContainerStarted","Data":"749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff"} Jul 24 13:55:16.866624 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:16.866585 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:55:16.866825 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:16.866807 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 13:55:16.866875 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:16.866850 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 13:55:17.867240 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:17.867208 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 13:55:17.867730 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:17.867265 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 13:55:18.866989 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:18.866952 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 13:55:18.867207 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:18.867013 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 13:55:19.736170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.736133 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr"] Jul 24 13:55:19.740633 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.740609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.743656 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.743635 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e01298-serving-cert\"" Jul 24 13:55:19.743788 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.743728 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-h9lpb\"" Jul 24 13:55:19.754405 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.754381 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr"] Jul 24 13:55:19.835909 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.835878 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.836112 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.835945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2msgd\" (UniqueName: \"kubernetes.io/projected/63ab1561-7078-443d-8317-7c680dcbfeae-kube-api-access-2msgd\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.836112 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.836089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ab1561-7078-443d-8317-7c680dcbfeae-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.866763 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.866728 2562 patch_prober.go:28] interesting pod/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 13:55:19.866951 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.866793 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 13:55:19.936777 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.936739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ab1561-7078-443d-8317-7c680dcbfeae-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.936976 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.936788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.936976 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.936841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2msgd\" (UniqueName: \"kubernetes.io/projected/63ab1561-7078-443d-8317-7c680dcbfeae-kube-api-access-2msgd\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.936976 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:55:19.936956 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-serving-cert: secret "maas-api-e2e-ipp-b-e01298-serving-cert" not found Jul 24 13:55:19.937150 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:55:19.937016 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls podName:63ab1561-7078-443d-8317-7c680dcbfeae nodeName:}" failed. No retries permitted until 2026-07-24 13:55:20.437000858 +0000 UTC m=+2747.036246047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls") pod "maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" (UID: "63ab1561-7078-443d-8317-7c680dcbfeae") : secret "maas-api-e2e-ipp-b-e01298-serving-cert" not found Jul 24 13:55:19.937467 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.937444 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ab1561-7078-443d-8317-7c680dcbfeae-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:19.945307 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:19.945282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2msgd\" (UniqueName: \"kubernetes.io/projected/63ab1561-7078-443d-8317-7c680dcbfeae-kube-api-access-2msgd\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:20.442107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.442075 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:20.444512 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.444490 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:20.654617 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.654575 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:20.797364 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.797331 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr"] Jul 24 13:55:20.798532 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:55:20.798497 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ab1561_7078_443d_8317_7c680dcbfeae.slice/crio-8776fc4363f200220c847865c609a48bcad8cb087d354aff1ff927a4f4386951 WatchSource:0}: Error finding container 8776fc4363f200220c847865c609a48bcad8cb087d354aff1ff927a4f4386951: Status 404 returned error can't find the container with id 8776fc4363f200220c847865c609a48bcad8cb087d354aff1ff927a4f4386951 Jul 24 13:55:20.800022 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.800003 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:55:20.870067 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.870040 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:55:20.870426 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.870409 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:55:20.871017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:20.871002 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:55:21.687434 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:21.687399 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" event={"ID":"63ab1561-7078-443d-8317-7c680dcbfeae","Type":"ContainerStarted","Data":"453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b"} Jul 24 13:55:21.687629 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:21.687454 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" event={"ID":"63ab1561-7078-443d-8317-7c680dcbfeae","Type":"ContainerStarted","Data":"8776fc4363f200220c847865c609a48bcad8cb087d354aff1ff927a4f4386951"} Jul 24 13:55:21.687629 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:21.687493 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:21.707363 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:21.707315 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" podStartSLOduration=2.342464771 podStartE2EDuration="2.707301058s" totalImagesPullingTime="364.836287ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:55:21.705463602 +0000 UTC m=+2748.304708813" watchObservedRunningTime="2026-07-24 13:55:21.707301058 +0000 UTC m=+2748.306546269" Jul 24 13:55:27.697650 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:27.697621 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:55:41.889780 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:55:41.889752 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-v4mkx_49addf5c-c5ab-446f-af41-7447ff877af5/payload-processing/0.log" Jul 24 13:56:39.559981 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:39.559953 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-v4mkx_49addf5c-c5ab-446f-af41-7447ff877af5/payload-processing/0.log" Jul 24 13:56:43.326340 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.326306 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg"] Jul 24 13:56:43.330170 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.330151 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.334558 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.334526 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-095476-serving-cert\"" Jul 24 13:56:43.339029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.339010 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg"] Jul 24 13:56:43.426421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.424573 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8944\" (UniqueName: \"kubernetes.io/projected/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-kube-api-access-b8944\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.426421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.424706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.426421 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.424739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.525912 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.525869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.526112 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.525968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8944\" (UniqueName: \"kubernetes.io/projected/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-kube-api-access-b8944\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.526112 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.526031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.526230 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:56:43.526132 2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-serving-cert: secret "maas-api-e2e-ipp-cleanup-095476-serving-cert" not found Jul 24 13:56:43.526230 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:56:43.526201 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls podName:3e55c6bc-77cc-458b-bcfd-ef5e07b18d21 nodeName:}" failed. No retries permitted until 2026-07-24 13:56:44.026179951 +0000 UTC m=+2830.625425144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" (UID: "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21") : secret "maas-api-e2e-ipp-cleanup-095476-serving-cert" not found Jul 24 13:56:43.526727 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.526708 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:43.535349 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:43.535329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8944\" (UniqueName: \"kubernetes.io/projected/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-kube-api-access-b8944\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:44.032408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.032369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:44.034915 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.034894 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:44.108637 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.108598 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb"] Jul 24 13:56:44.112168 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.112144 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.114494 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.114471 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-095476-dockercfg-kkzb4\"" Jul 24 13:56:44.114748 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.114732 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-095476\"" Jul 24 13:56:44.126671 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.126646 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb"] Jul 24 13:56:44.234818 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.234780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ff28cc9-e751-4635-968e-5d58045bf833-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.234988 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.234850 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf849\" (UniqueName: \"kubernetes.io/projected/3ff28cc9-e751-4635-968e-5d58045bf833-kube-api-access-mf849\") pod \"payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.241664 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.241644 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:44.336197 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.336165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ff28cc9-e751-4635-968e-5d58045bf833-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.336539 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.336248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf849\" (UniqueName: \"kubernetes.io/projected/3ff28cc9-e751-4635-968e-5d58045bf833-kube-api-access-mf849\") pod \"payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.337105 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.337084 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ff28cc9-e751-4635-968e-5d58045bf833-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.344814 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.344788 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf849\" (UniqueName: \"kubernetes.io/projected/3ff28cc9-e751-4635-968e-5d58045bf833-kube-api-access-mf849\") pod \"payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.384521 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.384498 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg"] Jul 24 13:56:44.386722 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:56:44.386697 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e55c6bc_77cc_458b_bcfd_ef5e07b18d21.slice/crio-8af492ce966283e4fc9fbd0358fe2e964efe726e9f581c1fe93a5ad9d2e9b7dc WatchSource:0}: Error finding container 8af492ce966283e4fc9fbd0358fe2e964efe726e9f581c1fe93a5ad9d2e9b7dc: Status 404 returned error can't find the container with id 8af492ce966283e4fc9fbd0358fe2e964efe726e9f581c1fe93a5ad9d2e9b7dc Jul 24 13:56:44.422828 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.422799 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:44.558896 ip-10-0-137-131 kubenswrapper[2562]: W0724 13:56:44.558865 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff28cc9_e751_4635_968e_5d58045bf833.slice/crio-f0bc394383202abec7b79543da5124a0713a625a208854b5c669f09d9a2c0650 WatchSource:0}: Error finding container f0bc394383202abec7b79543da5124a0713a625a208854b5c669f09d9a2c0650: Status 404 returned error can't find the container with id f0bc394383202abec7b79543da5124a0713a625a208854b5c669f09d9a2c0650 Jul 24 13:56:44.559564 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:44.559473 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb"] Jul 24 13:56:45.142017 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.141983 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" event={"ID":"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21","Type":"ContainerStarted","Data":"f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa"} Jul 24 13:56:45.142201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.142134 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:45.142201 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.142173 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" event={"ID":"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21","Type":"ContainerStarted","Data":"8af492ce966283e4fc9fbd0358fe2e964efe726e9f581c1fe93a5ad9d2e9b7dc"} Jul 24 13:56:45.143453 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.143430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" event={"ID":"3ff28cc9-e751-4635-968e-5d58045bf833","Type":"ContainerStarted","Data":"c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537"} Jul 24 13:56:45.143581 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.143468 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" event={"ID":"3ff28cc9-e751-4635-968e-5d58045bf833","Type":"ContainerStarted","Data":"f0bc394383202abec7b79543da5124a0713a625a208854b5c669f09d9a2c0650"} Jul 24 13:56:45.143581 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.143564 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:56:45.164351 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.164263 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" podStartSLOduration=1.6438277879999998 podStartE2EDuration="2.164250025s" totalImagesPullingTime="520.422237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:45.161729489 +0000 UTC m=+2831.760974714" watchObservedRunningTime="2026-07-24 13:56:45.164250025 +0000 UTC m=+2831.763495236" Jul 24 13:56:45.176137 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:45.176088 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" podStartSLOduration=1.176072252 podStartE2EDuration="1.176072252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:56:45.175720424 +0000 UTC m=+2831.774965634" watchObservedRunningTime="2026-07-24 13:56:45.176072252 +0000 UTC m=+2831.775317464" Jul 24 13:56:51.154602 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:51.154569 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:56:56.151314 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:56:56.151274 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" podUID="3ff28cc9-e751-4635-968e-5d58045bf833" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.61:9004: connect: connection refused" Jul 24 13:57:06.149799 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:06.149766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:57:19.306520 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.306485 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg"] Jul 24 13:57:19.307011 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.306799 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" podUID="3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" containerName="maas-api" containerID="cri-o://f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa" gracePeriod=30 Jul 24 13:57:19.367329 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.367301 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb"] Jul 24 13:57:19.367558 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.367515 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" podUID="3ff28cc9-e751-4635-968e-5d58045bf833" containerName="payload-pre-processing" containerID="cri-o://c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537" gracePeriod=30 Jul 24 13:57:19.641458 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.641430 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:57:19.674263 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.674243 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:57:19.780270 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780242 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf849\" (UniqueName: \"kubernetes.io/projected/3ff28cc9-e751-4635-968e-5d58045bf833-kube-api-access-mf849\") pod \"3ff28cc9-e751-4635-968e-5d58045bf833\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " Jul 24 13:57:19.780408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780330 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls\") pod \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " Jul 24 13:57:19.780408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780359 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ff28cc9-e751-4635-968e-5d58045bf833-plugins-config-volume\") pod \"3ff28cc9-e751-4635-968e-5d58045bf833\" (UID: \"3ff28cc9-e751-4635-968e-5d58045bf833\") " Jul 24 13:57:19.780408 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780389 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8944\" (UniqueName: \"kubernetes.io/projected/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-kube-api-access-b8944\") pod \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " Jul 24 13:57:19.780519 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780447 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-openshift-service-ca\") pod \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\" (UID: \"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21\") " Jul 24 13:57:19.780801 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780773 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff28cc9-e751-4635-968e-5d58045bf833-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ff28cc9-e751-4635-968e-5d58045bf833" (UID: "3ff28cc9-e751-4635-968e-5d58045bf833"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:19.780885 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.780860 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" (UID: "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:57:19.782378 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.782354 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" (UID: "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:57:19.782465 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.782408 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-kube-api-access-b8944" (OuterVolumeSpecName: "kube-api-access-b8944") pod "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" (UID: "3e55c6bc-77cc-458b-bcfd-ef5e07b18d21"). InnerVolumeSpecName "kube-api-access-b8944". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:19.782465 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.782415 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff28cc9-e751-4635-968e-5d58045bf833-kube-api-access-mf849" (OuterVolumeSpecName: "kube-api-access-mf849") pod "3ff28cc9-e751-4635-968e-5d58045bf833" (UID: "3ff28cc9-e751-4635-968e-5d58045bf833"). InnerVolumeSpecName "kube-api-access-mf849". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:57:19.882272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.882196 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-maas-api-tls\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:57:19.882272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.882224 2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ff28cc9-e751-4635-968e-5d58045bf833-plugins-config-volume\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:57:19.882272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.882235 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8944\" (UniqueName: \"kubernetes.io/projected/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-kube-api-access-b8944\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:57:19.882272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.882243 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21-openshift-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:57:19.882272 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:19.882252 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf849\" (UniqueName: \"kubernetes.io/projected/3ff28cc9-e751-4635-968e-5d58045bf833-kube-api-access-mf849\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:57:20.337560 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.337508 2562 generic.go:356] "Generic (PLEG): container finished" podID="3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" containerID="f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa" exitCode=0 Jul 24 13:57:20.337918 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.337568 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" Jul 24 13:57:20.337918 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.337582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" event={"ID":"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21","Type":"ContainerDied","Data":"f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa"} Jul 24 13:57:20.337918 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.337640 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg" event={"ID":"3e55c6bc-77cc-458b-bcfd-ef5e07b18d21","Type":"ContainerDied","Data":"8af492ce966283e4fc9fbd0358fe2e964efe726e9f581c1fe93a5ad9d2e9b7dc"} Jul 24 13:57:20.337918 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.337665 2562 scope.go:117] "RemoveContainer" containerID="f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa" Jul 24 13:57:20.339092 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.339070 2562 generic.go:356] "Generic (PLEG): container finished" podID="3ff28cc9-e751-4635-968e-5d58045bf833" containerID="c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537" exitCode=0 Jul 24 13:57:20.339211 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.339110 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" Jul 24 13:57:20.339286 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.339106 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" event={"ID":"3ff28cc9-e751-4635-968e-5d58045bf833","Type":"ContainerDied","Data":"c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537"} Jul 24 13:57:20.339286 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.339260 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb" event={"ID":"3ff28cc9-e751-4635-968e-5d58045bf833","Type":"ContainerDied","Data":"f0bc394383202abec7b79543da5124a0713a625a208854b5c669f09d9a2c0650"} Jul 24 13:57:20.347100 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.347082 2562 scope.go:117] "RemoveContainer" containerID="f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa" Jul 24 13:57:20.347346 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:57:20.347327 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa\": container with ID starting with f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa not found: ID does not exist" containerID="f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa" Jul 24 13:57:20.347424 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.347358 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa"} err="failed to get container status \"f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa\": rpc error: code = NotFound desc = could not find container \"f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa\": container with ID starting with f36fe01d40cf8745de6da41b3fcd7526c58b7409e8235c4d25233a1a23736caa not found: ID does not exist" Jul 24 13:57:20.347424 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.347381 2562 scope.go:117] "RemoveContainer" containerID="c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537" Jul 24 13:57:20.355878 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.355861 2562 scope.go:117] "RemoveContainer" containerID="c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537" Jul 24 13:57:20.356119 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:57:20.356096 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537\": container with ID starting with c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537 not found: ID does not exist" containerID="c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537" Jul 24 13:57:20.356188 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.356127 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537"} err="failed to get container status \"c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537\": rpc error: code = NotFound desc = could not find container \"c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537\": container with ID starting with c83a6eb6a8e25c5851c3b53a531921a6a29e86cd59b3efb1111e0ca1e7a97537 not found: ID does not exist" Jul 24 13:57:20.356321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.356304 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb"] Jul 24 13:57:20.360733 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.360714 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-095476-5f5dd855b4-nfkwb"] Jul 24 13:57:20.370826 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.370806 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg"] Jul 24 13:57:20.375159 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:20.375138 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-095476-6b56fbbc4c-4znzg"] Jul 24 13:57:21.896245 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:21.896214 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" path="/var/lib/kubelet/pods/3e55c6bc-77cc-458b-bcfd-ef5e07b18d21/volumes" Jul 24 13:57:21.896648 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:57:21.896632 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ff28cc9-e751-4635-968e-5d58045bf833" path="/var/lib/kubelet/pods/3ff28cc9-e751-4635-968e-5d58045bf833/volumes" Jul 24 13:58:57.412994 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:58:57.412918 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9"] Jul 24 13:58:57.413476 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:58:57.413151 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" containerID="cri-o://749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff" gracePeriod=30 Jul 24 13:59:02.659991 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.659967 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:59:02.752253 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752220 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-certs\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752263 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5r64\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-kube-api-access-q5r64\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752301 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-socket\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752323 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-credential-socket\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752365 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-token\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752442 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752422 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-podinfo\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752764 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752493 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-envoy\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752764 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752523 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7a70e-e388-48be-86f6-c67eae3e7019-istiod-ca-cert\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752764 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752583 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-data\") pod \"6bc7a70e-e388-48be-86f6-c67eae3e7019\" (UID: \"6bc7a70e-e388-48be-86f6-c67eae3e7019\") " Jul 24 13:59:02.752764 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752530 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:02.752764 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752624 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:02.753029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752885 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bc7a70e-e388-48be-86f6-c67eae3e7019-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:02.753029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752923 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.753029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752941 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.753029 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.752988 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:02.753441 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.753422 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:02.754746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.754717 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:02.754746 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.754723 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:59:02.754971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.754948 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-kube-api-access-q5r64" (OuterVolumeSpecName: "kube-api-access-q5r64") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "kube-api-access-q5r64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:02.754971 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.754957 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bc7a70e-e388-48be-86f6-c67eae3e7019" (UID: "6bc7a70e-e388-48be-86f6-c67eae3e7019"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:02.853516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853488 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5r64\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-kube-api-access-q5r64\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.853516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853513 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.853516 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853523 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.853735 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853531 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.853735 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853538 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.853735 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853567 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7a70e-e388-48be-86f6-c67eae3e7019-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.853735 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.853575 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7a70e-e388-48be-86f6-c67eae3e7019-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:02.885229 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.885198 2562 generic.go:356] "Generic (PLEG): container finished" podID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerID="749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff" exitCode=0 Jul 24 13:59:02.885427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.885246 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" Jul 24 13:59:02.885427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.885286 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" event={"ID":"6bc7a70e-e388-48be-86f6-c67eae3e7019","Type":"ContainerDied","Data":"749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff"} Jul 24 13:59:02.885427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.885337 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9" event={"ID":"6bc7a70e-e388-48be-86f6-c67eae3e7019","Type":"ContainerDied","Data":"f4b354b73a6ff90e9b5caf9f952019a762d0fedf590cfb4f41f03af5b385563e"} Jul 24 13:59:02.885427 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.885359 2562 scope.go:117] "RemoveContainer" containerID="749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff" Jul 24 13:59:02.894916 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.894900 2562 scope.go:117] "RemoveContainer" containerID="be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b" Jul 24 13:59:02.903635 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.903581 2562 scope.go:117] "RemoveContainer" containerID="749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff" Jul 24 13:59:02.903967 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:59:02.903932 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff\": container with ID starting with 749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff not found: ID does not exist" containerID="749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff" Jul 24 13:59:02.904061 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.903968 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff"} err="failed to get container status \"749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff\": rpc error: code = NotFound desc = could not find container \"749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff\": container with ID starting with 749b6e7bb5e0f58c18898d2936e6d401246f3d0c497038c34e7dcfb07b60a2ff not found: ID does not exist" Jul 24 13:59:02.904061 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.903991 2562 scope.go:117] "RemoveContainer" containerID="be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b" Jul 24 13:59:02.904307 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:59:02.904286 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b\": container with ID starting with be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b not found: ID does not exist" containerID="be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b" Jul 24 13:59:02.904351 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.904314 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b"} err="failed to get container status \"be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b\": rpc error: code = NotFound desc = could not find container \"be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b\": container with ID starting with be342d67f29ecc9d44c52c6f61dd65f1a7bbe8790c60039e0d8a7f5c984a573b not found: ID does not exist" Jul 24 13:59:02.906354 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.906336 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9"] Jul 24 13:59:02.909385 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:02.909366 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-e7750f-openshift-default-6496f867fd-bl9n9"] Jul 24 13:59:03.896635 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:03.896607 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" path="/var/lib/kubelet/pods/6bc7a70e-e388-48be-86f6-c67eae3e7019/volumes" Jul 24 13:59:08.377260 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.377230 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr"] Jul 24 13:59:08.377658 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.377434 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" podUID="63ab1561-7078-443d-8317-7c680dcbfeae" containerName="maas-api" containerID="cri-o://453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b" gracePeriod=30 Jul 24 13:59:08.640531 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.640473 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:59:08.705717 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.705688 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls\") pod \"63ab1561-7078-443d-8317-7c680dcbfeae\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " Jul 24 13:59:08.705875 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.705764 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2msgd\" (UniqueName: \"kubernetes.io/projected/63ab1561-7078-443d-8317-7c680dcbfeae-kube-api-access-2msgd\") pod \"63ab1561-7078-443d-8317-7c680dcbfeae\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " Jul 24 13:59:08.705875 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.705798 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ab1561-7078-443d-8317-7c680dcbfeae-openshift-service-ca\") pod \"63ab1561-7078-443d-8317-7c680dcbfeae\" (UID: \"63ab1561-7078-443d-8317-7c680dcbfeae\") " Jul 24 13:59:08.706207 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.706174 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ab1561-7078-443d-8317-7c680dcbfeae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63ab1561-7078-443d-8317-7c680dcbfeae" (UID: "63ab1561-7078-443d-8317-7c680dcbfeae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:08.707848 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.707826 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ab1561-7078-443d-8317-7c680dcbfeae-kube-api-access-2msgd" (OuterVolumeSpecName: "kube-api-access-2msgd") pod "63ab1561-7078-443d-8317-7c680dcbfeae" (UID: "63ab1561-7078-443d-8317-7c680dcbfeae"). InnerVolumeSpecName "kube-api-access-2msgd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:08.707930 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.707863 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63ab1561-7078-443d-8317-7c680dcbfeae" (UID: "63ab1561-7078-443d-8317-7c680dcbfeae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:59:08.807107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.807076 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ab1561-7078-443d-8317-7c680dcbfeae-openshift-service-ca\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.807107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.807105 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ab1561-7078-443d-8317-7c680dcbfeae-maas-api-tls\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.807107 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.807114 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2msgd\" (UniqueName: \"kubernetes.io/projected/63ab1561-7078-443d-8317-7c680dcbfeae-kube-api-access-2msgd\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:08.919321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.919237 2562 generic.go:356] "Generic (PLEG): container finished" podID="63ab1561-7078-443d-8317-7c680dcbfeae" containerID="453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b" exitCode=0 Jul 24 13:59:08.919321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.919283 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" Jul 24 13:59:08.919321 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.919307 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" event={"ID":"63ab1561-7078-443d-8317-7c680dcbfeae","Type":"ContainerDied","Data":"453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b"} Jul 24 13:59:08.919613 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.919346 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr" event={"ID":"63ab1561-7078-443d-8317-7c680dcbfeae","Type":"ContainerDied","Data":"8776fc4363f200220c847865c609a48bcad8cb087d354aff1ff927a4f4386951"} Jul 24 13:59:08.919613 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.919368 2562 scope.go:117] "RemoveContainer" containerID="453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b" Jul 24 13:59:08.928940 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.928922 2562 scope.go:117] "RemoveContainer" containerID="453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b" Jul 24 13:59:08.929189 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:59:08.929172 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b\": container with ID starting with 453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b not found: ID does not exist" containerID="453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b" Jul 24 13:59:08.929240 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.929198 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b"} err="failed to get container status \"453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b\": rpc error: code = NotFound desc = could not find container \"453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b\": container with ID starting with 453e8bd03362b0980c234cdca3e52773e300ad3a3fcc2acbf46d7afce0e6070b not found: ID does not exist" Jul 24 13:59:08.940189 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.940166 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr"] Jul 24 13:59:08.943653 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:08.943624 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e01298-6d9bb89cd9-ctgzr"] Jul 24 13:59:09.896172 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:09.896142 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63ab1561-7078-443d-8317-7c680dcbfeae" path="/var/lib/kubelet/pods/63ab1561-7078-443d-8317-7c680dcbfeae/volumes" Jul 24 13:59:41.172969 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:41.172928 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d"] Jul 24 13:59:41.173485 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:41.173180 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" containerID="cri-o://2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6" gracePeriod=30 Jul 24 13:59:46.421164 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.421137 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:59:46.545794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545714 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-token\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.545794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545750 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-data\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.545794 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545772 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istiod-ca-cert\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545805 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-certs\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545858 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-envoy\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545885 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-credential-socket\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545919 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-podinfo\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.545963 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-socket\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546056 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546014 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqjcm\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-kube-api-access-nqjcm\") pod \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\" (UID: \"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974\") " Jul 24 13:59:46.546373 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546171 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:46.546373 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546182 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:59:46.546373 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546192 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:46.546528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546438 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:46.546528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546441 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.546528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546484 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.546528 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.546501 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.547028 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.547002 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:46.548063 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.548037 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:46.548160 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.548091 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:59:46.548160 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.548096 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-kube-api-access-nqjcm" (OuterVolumeSpecName: "kube-api-access-nqjcm") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "kube-api-access-nqjcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:59:46.548160 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.548111 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" (UID: "0a148cc6-d3d7-45b6-91ee-1d2eee9b4974"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:59:46.647888 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.647864 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.647888 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.647883 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.647888 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.647891 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.648064 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.647899 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.648064 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.647906 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:46.648064 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:46.647915 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqjcm\" (UniqueName: \"kubernetes.io/projected/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974-kube-api-access-nqjcm\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 13:59:47.110665 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.110626 2562 generic.go:356] "Generic (PLEG): container finished" podID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerID="2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6" exitCode=0 Jul 24 13:59:47.110831 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.110674 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" Jul 24 13:59:47.110831 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.110705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" event={"ID":"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974","Type":"ContainerDied","Data":"2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6"} Jul 24 13:59:47.110831 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.110763 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d" event={"ID":"0a148cc6-d3d7-45b6-91ee-1d2eee9b4974","Type":"ContainerDied","Data":"92276421d55b4028fc991316a90e56e6dfc12c1f63577e5fe81d73ecd0255f50"} Jul 24 13:59:47.110831 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.110788 2562 scope.go:117] "RemoveContainer" containerID="2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6" Jul 24 13:59:47.120267 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.120246 2562 scope.go:117] "RemoveContainer" containerID="5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d" Jul 24 13:59:47.127978 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.127962 2562 scope.go:117] "RemoveContainer" containerID="2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6" Jul 24 13:59:47.128206 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:59:47.128184 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6\": container with ID starting with 2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6 not found: ID does not exist" containerID="2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6" Jul 24 13:59:47.128280 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.128221 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6"} err="failed to get container status \"2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6\": rpc error: code = NotFound desc = could not find container \"2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6\": container with ID starting with 2aac61315c50c47c1a33ad4621573c9e1992d3bce4822e3a5b77c6ae35f8f2a6 not found: ID does not exist" Jul 24 13:59:47.128280 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.128244 2562 scope.go:117] "RemoveContainer" containerID="5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d" Jul 24 13:59:47.128505 ip-10-0-137-131 kubenswrapper[2562]: E0724 13:59:47.128489 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d\": container with ID starting with 5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d not found: ID does not exist" containerID="5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d" Jul 24 13:59:47.128560 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.128511 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d"} err="failed to get container status \"5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d\": rpc error: code = NotFound desc = could not find container \"5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d\": container with ID starting with 5aa39b467257cf16103c7f5605d019d73c58b26ed018a15f7bdf664ef307c95d not found: ID does not exist" Jul 24 13:59:47.131995 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.131974 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d"] Jul 24 13:59:47.135847 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.135829 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-934481-openshift-default-7df6768488-vz89d"] Jul 24 13:59:47.896425 ip-10-0-137-131 kubenswrapper[2562]: I0724 13:59:47.896394 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" path="/var/lib/kubelet/pods/0a148cc6-d3d7-45b6-91ee-1d2eee9b4974/volumes" Jul 24 14:00:03.399909 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:03.399879 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj"] Jul 24 14:00:03.400346 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:03.400096 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" containerID="cri-o://77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98" gracePeriod=30 Jul 24 14:00:08.649945 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.649919 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 14:00:08.745171 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745136 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-data\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745186 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-228bt\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-kube-api-access-228bt\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745204 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-credential-socket\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745224 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-envoy\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745267 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-podinfo\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745298 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-certs\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745331 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-token\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745364 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745367 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-socket\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745772 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745392 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91086b03-c87c-44b2-b8bd-e2bac1294164-istiod-ca-cert\") pod \"91086b03-c87c-44b2-b8bd-e2bac1294164\" (UID: \"91086b03-c87c-44b2-b8bd-e2bac1294164\") " Jul 24 14:00:08.745772 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745467 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:00:08.745772 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745701 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:00:08.745772 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745734 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-credential-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.745916 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745791 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91086b03-c87c-44b2-b8bd-e2bac1294164-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 14:00:08.745916 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.745899 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:00:08.746223 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.746200 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-data" (OuterVolumeSpecName: "istio-data") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:00:08.747751 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.747722 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-token" (OuterVolumeSpecName: "istio-token") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:00:08.747864 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.747767 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 14:00:08.747864 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.747848 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 14:00:08.747979 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.747936 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-kube-api-access-228bt" (OuterVolumeSpecName: "kube-api-access-228bt") pod "91086b03-c87c-44b2-b8bd-e2bac1294164" (UID: "91086b03-c87c-44b2-b8bd-e2bac1294164"). InnerVolumeSpecName "kube-api-access-228bt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 14:00:08.846626 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846593 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-podinfo\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846626 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846617 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-certs\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846626 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846626 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-token\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846853 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846637 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-workload-socket\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846853 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846646 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91086b03-c87c-44b2-b8bd-e2bac1294164-istiod-ca-cert\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846853 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846654 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-data\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846853 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846662 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-228bt\" (UniqueName: \"kubernetes.io/projected/91086b03-c87c-44b2-b8bd-e2bac1294164-kube-api-access-228bt\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:08.846853 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:08.846672 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91086b03-c87c-44b2-b8bd-e2bac1294164-istio-envoy\") on node \"ip-10-0-137-131.ec2.internal\" DevicePath \"\"" Jul 24 14:00:09.233163 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.233131 2562 generic.go:356] "Generic (PLEG): container finished" podID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerID="77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98" exitCode=0 Jul 24 14:00:09.233329 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.233181 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" Jul 24 14:00:09.233329 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.233203 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" event={"ID":"91086b03-c87c-44b2-b8bd-e2bac1294164","Type":"ContainerDied","Data":"77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98"} Jul 24 14:00:09.233329 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.233254 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj" event={"ID":"91086b03-c87c-44b2-b8bd-e2bac1294164","Type":"ContainerDied","Data":"15aff688b07e4d02f99166ebe5ae9ff5745e4a58234222ef8bdb272712eb2d3a"} Jul 24 14:00:09.233329 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.233284 2562 scope.go:117] "RemoveContainer" containerID="77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98" Jul 24 14:00:09.261916 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.243029 2562 scope.go:117] "RemoveContainer" containerID="587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200" Jul 24 14:00:09.261916 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.257948 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj"] Jul 24 14:00:09.261916 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.261707 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0a0df7-openshift-default-66479dd96-7x7mj"] Jul 24 14:00:09.274387 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.274367 2562 scope.go:117] "RemoveContainer" containerID="77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98" Jul 24 14:00:09.274696 ip-10-0-137-131 kubenswrapper[2562]: E0724 14:00:09.274676 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98\": container with ID starting with 77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98 not found: ID does not exist" containerID="77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98" Jul 24 14:00:09.274771 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.274705 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98"} err="failed to get container status \"77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98\": rpc error: code = NotFound desc = could not find container \"77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98\": container with ID starting with 77ef9827d1bec1a6a4216e4044c39c485025bae578bc360ebddb31f4260a5a98 not found: ID does not exist" Jul 24 14:00:09.274771 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.274725 2562 scope.go:117] "RemoveContainer" containerID="587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200" Jul 24 14:00:09.275000 ip-10-0-137-131 kubenswrapper[2562]: E0724 14:00:09.274981 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200\": container with ID starting with 587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200 not found: ID does not exist" containerID="587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200" Jul 24 14:00:09.275045 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.275007 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200"} err="failed to get container status \"587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200\": rpc error: code = NotFound desc = could not find container \"587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200\": container with ID starting with 587b59dba22c3a217e5627fa7401e504d5256cff4d5b0dbc1615e6a5fa32d200 not found: ID does not exist" Jul 24 14:00:09.895935 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:09.895905 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" path="/var/lib/kubelet/pods/91086b03-c87c-44b2-b8bd-e2bac1294164/volumes" Jul 24 14:00:10.859032 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:10.859005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cf5655576-4xrjx_260770a9-bdf6-4a31-b462-ae4f4f3b0d97/manager/0.log" Jul 24 14:00:11.496153 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:11.496125 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-gk7vk_c2a56fe7-4957-4e9e-91df-db4c570b7535/copy-manifests/0.log" Jul 24 14:00:11.508207 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:11.508178 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-gk7vk_c2a56fe7-4957-4e9e-91df-db4c570b7535/manager/0.log" Jul 24 14:00:11.901070 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:11.900998 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54b587c679-mcfgw_b551d8fb-725b-4988-a4c1-5b4df7cfa981/maas-api/0.log" Jul 24 14:00:13.238656 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.238623 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p_98f94866-c253-4b6b-8c0c-878220778359/util/0.log" Jul 24 14:00:13.244686 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.244663 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p_98f94866-c253-4b6b-8c0c-878220778359/pull/0.log" Jul 24 14:00:13.250407 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.250392 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p_98f94866-c253-4b6b-8c0c-878220778359/extract/0.log" Jul 24 14:00:13.373411 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.373383 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn_760249a8-7a07-496c-a5a0-c4f147034025/util/0.log" Jul 24 14:00:13.379534 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.379515 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn_760249a8-7a07-496c-a5a0-c4f147034025/pull/0.log" Jul 24 14:00:13.385824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.385807 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn_760249a8-7a07-496c-a5a0-c4f147034025/extract/0.log" Jul 24 14:00:13.502979 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.502901 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc_9806a155-10e3-4b57-a897-d9443c7b68a4/extract/0.log" Jul 24 14:00:13.508571 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.508535 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc_9806a155-10e3-4b57-a897-d9443c7b68a4/util/0.log" Jul 24 14:00:13.514716 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.514695 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc_9806a155-10e3-4b57-a897-d9443c7b68a4/pull/0.log" Jul 24 14:00:13.640439 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.640410 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f_8d073917-4713-459e-9960-b177cacae466/util/0.log" Jul 24 14:00:13.646434 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.646415 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f_8d073917-4713-459e-9960-b177cacae466/pull/0.log" Jul 24 14:00:13.652467 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:13.652445 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f_8d073917-4713-459e-9960-b177cacae466/extract/0.log" Jul 24 14:00:15.080513 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:15.080486 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh_9589e305-2fff-4be8-b990-a7e1debb86da/istio-proxy/0.log" Jul 24 14:00:15.205652 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:15.205629 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bmrt_c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef/discovery/0.log" Jul 24 14:00:15.575019 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:15.574987 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/1.log" Jul 24 14:00:15.701935 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:15.701910 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4tzvz_3edbf7e3-9350-4ede-a85e-8fccbf657c1e/payload-pre-processing/0.log" Jul 24 14:00:15.835672 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:15.835598 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-v4mkx_49addf5c-c5ab-446f-af41-7447ff877af5/payload-processing/0.log" Jul 24 14:00:16.581792 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:16.581765 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-qclg6_70c03d51-9e6b-43ae-9289-3cfc6a2c0eeb/main/0.log" Jul 24 14:00:16.826621 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:16.826596 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-v8lfh_52e9ef94-e95a-4a9d-871e-da1ddce29ca6/main/0.log" Jul 24 14:00:22.789424 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789389 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mgcsj/must-gather-q2s9k"] Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789741 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789752 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789763 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789768 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789774 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789780 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789791 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" containerName="maas-api" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789797 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" containerName="maas-api" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789806 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789811 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789817 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63ab1561-7078-443d-8317-7c680dcbfeae" containerName="maas-api" Jul 24 14:00:22.789824 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789821 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ab1561-7078-443d-8317-7c680dcbfeae" containerName="maas-api" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789833 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff28cc9-e751-4635-968e-5d58045bf833" containerName="payload-pre-processing" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789838 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff28cc9-e751-4635-968e-5d58045bf833" containerName="payload-pre-processing" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789847 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789852 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789903 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e55c6bc-77cc-458b-bcfd-ef5e07b18d21" containerName="maas-api" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789910 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="63ab1561-7078-443d-8317-7c680dcbfeae" containerName="maas-api" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789917 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789923 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789929 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789935 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789940 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc7a70e-e388-48be-86f6-c67eae3e7019" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789945 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a148cc6-d3d7-45b6-91ee-1d2eee9b4974" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789952 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff28cc9-e751-4635-968e-5d58045bf833" containerName="payload-pre-processing" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.789998 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" Jul 24 14:00:22.790210 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.790003 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="91086b03-c87c-44b2-b8bd-e2bac1294164" containerName="istio-proxy" Jul 24 14:00:22.793089 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.793070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:22.795455 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.795430 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mgcsj\"/\"default-dockercfg-twfrm\"" Jul 24 14:00:22.795455 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.795449 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mgcsj\"/\"kube-root-ca.crt\"" Jul 24 14:00:22.795717 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.795517 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mgcsj\"/\"openshift-service-ca.crt\"" Jul 24 14:00:22.803669 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.803644 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mgcsj/must-gather-q2s9k"] Jul 24 14:00:22.856116 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.856090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmcwm\" (UniqueName: \"kubernetes.io/projected/f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982-kube-api-access-nmcwm\") pod \"must-gather-q2s9k\" (UID: \"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982\") " pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:22.856273 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.856133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982-must-gather-output\") pod \"must-gather-q2s9k\" (UID: \"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982\") " pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:22.956560 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.956512 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmcwm\" (UniqueName: \"kubernetes.io/projected/f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982-kube-api-access-nmcwm\") pod \"must-gather-q2s9k\" (UID: \"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982\") " pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:22.956694 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.956602 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982-must-gather-output\") pod \"must-gather-q2s9k\" (UID: \"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982\") " pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:22.956984 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.956966 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982-must-gather-output\") pod \"must-gather-q2s9k\" (UID: \"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982\") " pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:22.966199 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:22.966172 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmcwm\" (UniqueName: \"kubernetes.io/projected/f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982-kube-api-access-nmcwm\") pod \"must-gather-q2s9k\" (UID: \"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982\") " pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:23.103275 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:23.103193 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mgcsj/must-gather-q2s9k" Jul 24 14:00:23.234420 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:23.234395 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mgcsj/must-gather-q2s9k"] Jul 24 14:00:23.236298 ip-10-0-137-131 kubenswrapper[2562]: W0724 14:00:23.236274 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9eb1b8b_84bb_4d4c_93d7_b3d56e05b982.slice/crio-74952e9942606c3516e91bb53102e5a14ba12aef27742cef9d0462602e0f586c WatchSource:0}: Error finding container 74952e9942606c3516e91bb53102e5a14ba12aef27742cef9d0462602e0f586c: Status 404 returned error can't find the container with id 74952e9942606c3516e91bb53102e5a14ba12aef27742cef9d0462602e0f586c Jul 24 14:00:23.238646 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:23.238625 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 14:00:23.304674 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:23.304645 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/must-gather-q2s9k" event={"ID":"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982","Type":"ContainerStarted","Data":"74952e9942606c3516e91bb53102e5a14ba12aef27742cef9d0462602e0f586c"} Jul 24 14:00:25.319756 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:25.319698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/must-gather-q2s9k" event={"ID":"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982","Type":"ContainerStarted","Data":"b98e1ce8943b2fef976e661b63e5c3e6f9644c728b7459196a6a2c2e32c606a6"} Jul 24 14:00:25.319756 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:25.319755 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/must-gather-q2s9k" event={"ID":"f9eb1b8b-84bb-4d4c-93d7-b3d56e05b982","Type":"ContainerStarted","Data":"3585ddb34441f33dfdcea3e57284ea181be85cd43e73dba4c374600f76f6d418"} Jul 24 14:00:25.339072 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:25.339020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mgcsj/must-gather-q2s9k" podStartSLOduration=2.019653333 podStartE2EDuration="3.339005348s" totalImagesPullingTime="1.319352015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 14:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:00:25.336621109 +0000 UTC m=+3051.935866320" watchObservedRunningTime="2026-07-24 14:00:25.339005348 +0000 UTC m=+3051.938250559" Jul 24 14:00:26.159129 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:26.159082 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dv8tx_277edfa4-9e9a-4177-b38d-1bdffa4457ed/global-pull-secret-syncer/0.log" Jul 24 14:00:26.284066 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:26.284033 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xkb2z_0c393e7a-a2e8-416d-91e8-b18832ac181b/konnectivity-agent/0.log" Jul 24 14:00:26.328858 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:26.328826 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-131.ec2.internal_be7a3e104963d2d6fa190dbf68a99d50/haproxy/0.log" Jul 24 14:00:30.971120 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:30.971064 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p_98f94866-c253-4b6b-8c0c-878220778359/extract/0.log" Jul 24 14:00:31.004379 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.004296 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p_98f94866-c253-4b6b-8c0c-878220778359/util/0.log" Jul 24 14:00:31.027810 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.027785 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759ztw8p_98f94866-c253-4b6b-8c0c-878220778359/pull/0.log" Jul 24 14:00:31.055855 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.055828 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn_760249a8-7a07-496c-a5a0-c4f147034025/extract/0.log" Jul 24 14:00:31.087215 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.087191 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn_760249a8-7a07-496c-a5a0-c4f147034025/util/0.log" Jul 24 14:00:31.122338 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.122300 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06skmn_760249a8-7a07-496c-a5a0-c4f147034025/pull/0.log" Jul 24 14:00:31.167567 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.167522 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc_9806a155-10e3-4b57-a897-d9443c7b68a4/extract/0.log" Jul 24 14:00:31.207957 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.207920 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc_9806a155-10e3-4b57-a897-d9443c7b68a4/util/0.log" Jul 24 14:00:31.234614 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.234390 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73xrdkc_9806a155-10e3-4b57-a897-d9443c7b68a4/pull/0.log" Jul 24 14:00:31.271566 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.271293 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f_8d073917-4713-459e-9960-b177cacae466/extract/0.log" Jul 24 14:00:31.299387 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.299346 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f_8d073917-4713-459e-9960-b177cacae466/util/0.log" Jul 24 14:00:31.323773 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:31.323742 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1frh8f_8d073917-4713-459e-9960-b177cacae466/pull/0.log" Jul 24 14:00:33.960849 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:33.960817 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/alertmanager/0.log" Jul 24 14:00:33.986167 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:33.986137 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/config-reloader/0.log" Jul 24 14:00:34.008739 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.008713 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/kube-rbac-proxy-web/0.log" Jul 24 14:00:34.032176 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.032151 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/kube-rbac-proxy/0.log" Jul 24 14:00:34.055837 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.055811 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/kube-rbac-proxy-metric/0.log" Jul 24 14:00:34.079568 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.079529 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/prom-label-proxy/0.log" Jul 24 14:00:34.110207 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.110175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a4fa6fdf-b2e8-414b-826b-b7cb084a6194/init-config-reloader/0.log" Jul 24 14:00:34.194217 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.194194 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-m6brm_495efb19-5d76-45a8-9f29-3dfca443e378/kube-state-metrics/0.log" Jul 24 14:00:34.219602 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.219517 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-m6brm_495efb19-5d76-45a8-9f29-3dfca443e378/kube-rbac-proxy-main/0.log" Jul 24 14:00:34.245621 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.245568 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-m6brm_495efb19-5d76-45a8-9f29-3dfca443e378/kube-rbac-proxy-self/0.log" Jul 24 14:00:34.413719 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.413690 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h55pl_e09e824f-b26a-44b8-aff1-c7f2caf46b9a/node-exporter/0.log" Jul 24 14:00:34.436339 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.436314 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h55pl_e09e824f-b26a-44b8-aff1-c7f2caf46b9a/kube-rbac-proxy/0.log" Jul 24 14:00:34.457826 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.457797 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h55pl_e09e824f-b26a-44b8-aff1-c7f2caf46b9a/init-textfile/0.log" Jul 24 14:00:34.555481 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.555407 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-pzpv5_14a76ed7-c3d3-4843-9f7b-3f0619399d57/kube-rbac-proxy-main/0.log" Jul 24 14:00:34.579671 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.579641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-pzpv5_14a76ed7-c3d3-4843-9f7b-3f0619399d57/kube-rbac-proxy-self/0.log" Jul 24 14:00:34.603413 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:34.603370 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-pzpv5_14a76ed7-c3d3-4843-9f7b-3f0619399d57/openshift-state-metrics/0.log" Jul 24 14:00:35.515905 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.515865 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9"] Jul 24 14:00:35.520345 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.520315 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.531181 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.531154 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9"] Jul 24 14:00:35.593566 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.589009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-proc\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.593566 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.589093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-sys\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.593566 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.589142 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-podres\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.593566 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.589197 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-lib-modules\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.593566 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.589240 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7chj9\" (UniqueName: \"kubernetes.io/projected/a29f238c-71f3-4e13-ab55-c64ce94a1cee-kube-api-access-7chj9\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.689994 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.689960 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-podres\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690172 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-lib-modules\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690172 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690088 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7chj9\" (UniqueName: \"kubernetes.io/projected/a29f238c-71f3-4e13-ab55-c64ce94a1cee-kube-api-access-7chj9\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690172 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690131 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-podres\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690172 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-proc\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690360 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690198 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-proc\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690360 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-sys\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690360 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690217 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-lib-modules\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.690360 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.690313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a29f238c-71f3-4e13-ab55-c64ce94a1cee-sys\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.697325 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.697300 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7chj9\" (UniqueName: \"kubernetes.io/projected/a29f238c-71f3-4e13-ab55-c64ce94a1cee-kube-api-access-7chj9\") pod \"perf-node-gather-daemonset-q9bm9\" (UID: \"a29f238c-71f3-4e13-ab55-c64ce94a1cee\") " pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:35.834077 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:35.833974 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:36.005267 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:36.005235 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9"] Jul 24 14:00:36.404425 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:36.404340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" event={"ID":"a29f238c-71f3-4e13-ab55-c64ce94a1cee","Type":"ContainerStarted","Data":"8c6e118d0136cedefc2c1ed71c51aad4818aa93142b87c3289152c791dc73809"} Jul 24 14:00:36.404425 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:36.404405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" event={"ID":"a29f238c-71f3-4e13-ab55-c64ce94a1cee","Type":"ContainerStarted","Data":"3bc00690befe6a30bd306aa1c2dd2ca1d831733bb4da4c39b88c24bdd74eedeb"} Jul 24 14:00:36.404674 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:36.404451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:36.422084 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:36.422025 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" podStartSLOduration=1.422005506 podStartE2EDuration="1.422005506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 14:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 14:00:36.418940129 +0000 UTC m=+3063.018185340" watchObservedRunningTime="2026-07-24 14:00:36.422005506 +0000 UTC m=+3063.021250718" Jul 24 14:00:39.006669 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:39.006633 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4pgm_40a02687-5781-4843-8152-083a322887c0/dns/0.log" Jul 24 14:00:39.029288 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:39.029263 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4pgm_40a02687-5781-4843-8152-083a322887c0/kube-rbac-proxy/0.log" Jul 24 14:00:39.106311 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:39.106280 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g5525_86f4acca-d628-4c62-986a-f66e371b9aa4/dns-node-resolver/0.log" Jul 24 14:00:39.674424 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:39.674397 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dd4f6b6c8-zhkv2_87c96c12-0d56-4d4c-b3b5-623198e1a45d/registry/0.log" Jul 24 14:00:39.695679 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:39.695657 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2z5z2_977bf748-3237-46dd-8171-f882894b111d/node-ca/0.log" Jul 24 14:00:40.681847 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:40.681819 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh_9589e305-2fff-4be8-b990-a7e1debb86da/istio-proxy/0.log" Jul 24 14:00:40.868791 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:40.868744 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bmrt_c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef/discovery/0.log" Jul 24 14:00:40.944558 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:40.944461 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/0.log" Jul 24 14:00:40.979366 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:40.979337 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/1.log" Jul 24 14:00:41.014437 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:41.014400 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4tzvz_3edbf7e3-9350-4ede-a85e-8fccbf657c1e/payload-pre-processing/0.log" Jul 24 14:00:41.045285 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:41.045259 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-v4mkx_49addf5c-c5ab-446f-af41-7447ff877af5/payload-processing/0.log" Jul 24 14:00:41.941316 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:41.941277 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xk4j5_2d14cd9d-a3ef-46c4-abd7-b334b0f0afb2/serve-healthcheck-canary/0.log" Jul 24 14:00:42.423189 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:42.423105 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mgcsj/perf-node-gather-daemonset-q9bm9" Jul 24 14:00:42.517811 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:42.517784 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sn7tj_7586cdf8-d5f7-4c83-81d5-73d6de053f9b/kube-rbac-proxy/0.log" Jul 24 14:00:42.538659 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:42.538627 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sn7tj_7586cdf8-d5f7-4c83-81d5-73d6de053f9b/exporter/0.log" Jul 24 14:00:42.561795 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:42.561772 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sn7tj_7586cdf8-d5f7-4c83-81d5-73d6de053f9b/extractor/0.log" Jul 24 14:00:45.691589 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:45.691304 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6cf5655576-4xrjx_260770a9-bdf6-4a31-b462-ae4f4f3b0d97/manager/0.log" Jul 24 14:00:46.022392 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.022313 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh_9589e305-2fff-4be8-b990-a7e1debb86da/istio-proxy/0.log" Jul 24 14:00:46.045426 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.045395 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-gk7vk_c2a56fe7-4957-4e9e-91df-db4c570b7535/manager/0.log" Jul 24 14:00:46.062812 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.062786 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dbfd75c57-gk7vk_c2a56fe7-4957-4e9e-91df-db4c570b7535/copy-manifests/0.log" Jul 24 14:00:46.202084 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.202059 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bmrt_c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef/discovery/0.log" Jul 24 14:00:46.252737 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.252704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/0.log" Jul 24 14:00:46.292733 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.292651 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/1.log" Jul 24 14:00:46.319668 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.319642 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4tzvz_3edbf7e3-9350-4ede-a85e-8fccbf657c1e/payload-pre-processing/0.log" Jul 24 14:00:46.343184 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:46.343154 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-v4mkx_49addf5c-c5ab-446f-af41-7447ff877af5/payload-processing/0.log" Jul 24 14:00:47.417035 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:47.417008 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5d6cbddddd-v4bql_8ecea6c0-8d5c-4903-8070-aad08d9583c7/manager/0.log" Jul 24 14:00:47.813291 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:47.813243 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-rfvlh_9589e305-2fff-4be8-b990-a7e1debb86da/istio-proxy/0.log" Jul 24 14:00:47.987106 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:47.987079 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8bmrt_c9462b9c-63dd-4327-8bcb-f6fe0b4b24ef/discovery/0.log" Jul 24 14:00:48.035493 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:48.035463 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/0.log" Jul 24 14:00:48.075103 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:48.075020 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-kcjwr_b7de505d-0a1c-4d53-9bc6-79eea2973b8e/istio-proxy/1.log" Jul 24 14:00:48.096307 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:48.096260 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-4tzvz_3edbf7e3-9350-4ede-a85e-8fccbf657c1e/payload-pre-processing/0.log" Jul 24 14:00:48.119279 ip-10-0-137-131 kubenswrapper[2562]: I0724 14:00:48.119246 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-v4mkx_49addf5c-c5ab-446f-af41-7447ff877af5/payload-processing/0.log"